[PATCH V2 1/2] tools/perf/tests: perf all metrics test fails when perf_event access is restricted

[PATCH V2 1/2] tools/perf/tests: perf all metrics test fails when perf_event access is restricted

[Athira Rajeev]

Perf all metrics test fails as below when perf_event access
is restricted.

   ./perf test -v "perf all metrics test"
   Metric 'Memory_RD_BW_Chip' not printed in:
   Error:
   Access to performance monitoring and observability operations is limited.
   Enforced MAC policy settings (SELinux) can limit access to performance
   —
   access to performance monitoring and observability operations for processes
   without CAP_PERFMON, CAP_SYS_PTRACE or CAP_SYS_ADMIN Linux capability.
   —
   test child finished with -1
   ---- end ----
   perf all metrics test: FAILED!

The perf all metrics test picks the input events from
"perf list --raw-dump metrics" and runs "perf stat -M "$m""
for each of the metrics in the list. It fails here for some
of the metrics which needs access, since it collects system
wide resource details/statistics. Fix the testcase to skip
those metric events. The check is added at two places:
- When metric is run for workload monitoring
- when metric is run for system wide.

It could happen that some of the metric events are not valid
in per-thread mode. Example from an x86 system:

    $ ./perf stat -M smi_cycles true
    Error:
    Invalid event (msr/smi/u) in per-thread mode, enable system wide with '-a'.

The test fallbacks to system wide if first stage fails.
But when run with system wide for this metric, it hits the
issue with access restriction.

    $ ./perf stat -M smi_cycles -a sleep 0.1
    Error:
    Access to performance monitoring and observability operations is limited.

Similar some events report not supported while running with system wide.

Example from an x86 system:

   $ ./perf stat -M tma_info_system_socket_clks -a sleep 0.1

   Performance counter stats for 'system wide':

    <not supported>      cbox_0/event=0x0/u

       0.102633747 seconds time elapsed

Hence the checks for unsupported events and access restrictions is
added for both cases.

Signed-off-by: Athira Rajeev <atrajeev@linux.vnet.ibm.com>
---
Changelog:
v1 -> v2:
 Added the check for access restriction in workload as
 well as system wide check.

 tools/perf/tests/shell/stat_all_metrics.sh | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/tools/perf/tests/shell/stat_all_metrics.sh b/tools/perf/tests/shell/stat_all_metrics.sh
index 54774525e18a..b26420885560 100755
--- a/tools/perf/tests/shell/stat_all_metrics.sh
+++ b/tools/perf/tests/shell/stat_all_metrics.sh
@@ -1,18 +1,18 @@
 #!/bin/bash
 # perf all metrics test
 # SPDX-License-Identifier: GPL-2.0
-
 err=0
+//list="tma_info_system_socket_clks"
 for m in $(perf list --raw-dump metrics); do
   echo "Testing $m"
   result=$(perf stat -M "$m" true 2>&1)
-  if [[ "$result" =~ ${m:0:50} ]] || [[ "$result" =~ "<not supported>" ]]
+  if [[ "$result" =~ ${m:0:50} ]] || [[ "$result" =~ "<not supported>" ]] || [[ "$result" =~ "Access to performance monitoring and observability operations is limited" ]]
   then
     continue
   fi
   # Failed so try system wide.
   result=$(perf stat -M "$m" -a sleep 0.01 2>&1)
-  if [[ "$result" =~ ${m:0:50} ]]
+  if [[ "$result" =~ ${m:0:50} ]] || [[ "$result" =~ "<not supported>" ]] ||  [[ "$result" =~ "Access to performance monitoring and observability operations is limited" ]]
   then
     continue
   fi
-- 
2.31.1

[PATCH V2 1/2] tools/perf/tests: perf all metrics test fails when perf_event access is restricted

[Athira Rajeev]

Perf all metrics test fails as below when perf_event access
is restricted.

   ./perf test -v "perf all metrics test"
   Metric 'Memory_RD_BW_Chip' not printed in:
   Error:
   Access to performance monitoring and observability operations is limited.
   Enforced MAC policy settings (SELinux) can limit access to performance
   —
   access to performance monitoring and observability operations for processes
   without CAP_PERFMON, CAP_SYS_PTRACE or CAP_SYS_ADMIN Linux capability.
   —
   test child finished with -1
   ---- end ----
   perf all metrics test: FAILED!

The perf all metrics test picks the input events from
"perf list --raw-dump metrics" and runs "perf stat -M "$m""
for each of the metrics in the list. It fails here for some
of the metrics which needs access, since it collects system
wide resource details/statistics. Fix the testcase to skip
those metric events. The check is added at two places:
- When metric is run for workload monitoring
- when metric is run for system wide.

It could happen that some of the metric events are not valid
in per-thread mode. Example from an x86 system:

    $ ./perf stat -M smi_cycles true
    Error:
    Invalid event (msr/smi/u) in per-thread mode, enable system wide with '-a'.

The test fallbacks to system wide if first stage fails.
But when run with system wide for this metric, it hits the
issue with access restriction.

    $ ./perf stat -M smi_cycles -a sleep 0.1
    Error:
    Access to performance monitoring and observability operations is limited.

Similar some events report not supported while running with system wide.

Example from an x86 system:

   $ ./perf stat -M tma_info_system_socket_clks -a sleep 0.1

   Performance counter stats for 'system wide':

    <not supported>      cbox_0/event=0x0/u

       0.102633747 seconds time elapsed

Hence the checks for unsupported events and access restrictions is
added for both cases.

Signed-off-by: Athira Rajeev <atrajeev@linux.vnet.ibm.com>
---
Changelog:
v1 -> v2:
 Added the check for access restriction in workload as
 well as system wide check.

 tools/perf/tests/shell/stat_all_metrics.sh | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/tools/perf/tests/shell/stat_all_metrics.sh b/tools/perf/tests/shell/stat_all_metrics.sh
index 54774525e18a..b26420885560 100755
--- a/tools/perf/tests/shell/stat_all_metrics.sh
+++ b/tools/perf/tests/shell/stat_all_metrics.sh
@@ -1,18 +1,18 @@
 #!/bin/bash
 # perf all metrics test
 # SPDX-License-Identifier: GPL-2.0
-
 err=0
+//list="tma_info_system_socket_clks"
 for m in $(perf list --raw-dump metrics); do
   echo "Testing $m"
   result=$(perf stat -M "$m" true 2>&1)
-  if [[ "$result" =~ ${m:0:50} ]] || [[ "$result" =~ "<not supported>" ]]
+  if [[ "$result" =~ ${m:0:50} ]] || [[ "$result" =~ "<not supported>" ]] || [[ "$result" =~ "Access to performance monitoring and observability operations is limited" ]]
   then
     continue
   fi
   # Failed so try system wide.
   result=$(perf stat -M "$m" -a sleep 0.01 2>&1)
-  if [[ "$result" =~ ${m:0:50} ]]
+  if [[ "$result" =~ ${m:0:50} ]] || [[ "$result" =~ "<not supported>" ]] ||  [[ "$result" =~ "Access to performance monitoring and observability operations is limited" ]]
   then
     continue
   fi
-- 
2.31.1

[PATCH V2 2/2] tools/perf/tests: perf all metricgroups test fails when perf_event access is restricted

[Athira Rajeev]

Perf all metricgroups test fails as below when perf_event access
is restricted.

    ./perf test -v "perf all metricgroups test"
    Testing Memory_BW
    Error:
    Access to performance monitoring and observability operations is limited.
    Enforced MAC policy settings (SELinux) can limit access to performance
    access to performance monitoring and observability operations for processes
    without CAP_PERFMON, CAP_SYS_PTRACE or CAP_SYS_ADMIN Linux capability.

    test child finished with -1
    ---- end ----
    perf all metricgroups test: FAILED!

Fix the testcase to skip those metric events which needs perf_event access
explicitly. The exit code of the testcase is based on return code of
the perf stat command ( enabled by set -e option ). Hence save the
exit status in a variable and use that to decide success or fail for the
testcase.

Signed-off-by: Athira Rajeev <atrajeev@linux.vnet.ibm.com>
---
Changelog:
v1 -> v2:
 Changed the condition to use "echo" and "grep" so it works on
 Posix shell as well.

 tools/perf/tests/shell/stat_all_metricgroups.sh | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)

diff --git a/tools/perf/tests/shell/stat_all_metricgroups.sh b/tools/perf/tests/shell/stat_all_metricgroups.sh
index cb35e488809a..eaa5e1172294 100755
--- a/tools/perf/tests/shell/stat_all_metricgroups.sh
+++ b/tools/perf/tests/shell/stat_all_metricgroups.sh
@@ -2,11 +2,19 @@
 # perf all metricgroups test
 # SPDX-License-Identifier: GPL-2.0
 
-set -e
-
 for m in $(perf list --raw-dump metricgroups); do
   echo "Testing $m"
-  perf stat -M "$m" -a true
+  result=$(perf stat -M "$m" -a true 2>&1)
+  rc=$?
+  # Skip if there is no access to perf_events monitoring
+  # Otherwise exit based on the return code of perf comamnd.
+  if echo "$result" | grep -q "Access to performance monitoring and observability operations is limited";
+  then
+      continue
+  else
+      [ $rc -ne 0 ] && exit $rc
+  fi
+
 done
 
 exit 0
-- 
2.31.1

[PATCH V2 2/2] tools/perf/tests: perf all metricgroups test fails when perf_event access is restricted

[Athira Rajeev]

Perf all metricgroups test fails as below when perf_event access
is restricted.

    ./perf test -v "perf all metricgroups test"
    Testing Memory_BW
    Error:
    Access to performance monitoring and observability operations is limited.
    Enforced MAC policy settings (SELinux) can limit access to performance
    access to performance monitoring and observability operations for processes
    without CAP_PERFMON, CAP_SYS_PTRACE or CAP_SYS_ADMIN Linux capability.

    test child finished with -1
    ---- end ----
    perf all metricgroups test: FAILED!

Fix the testcase to skip those metric events which needs perf_event access
explicitly. The exit code of the testcase is based on return code of
the perf stat command ( enabled by set -e option ). Hence save the
exit status in a variable and use that to decide success or fail for the
testcase.

Signed-off-by: Athira Rajeev <atrajeev@linux.vnet.ibm.com>
---
Changelog:
v1 -> v2:
 Changed the condition to use "echo" and "grep" so it works on
 Posix shell as well.

 tools/perf/tests/shell/stat_all_metricgroups.sh | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)

diff --git a/tools/perf/tests/shell/stat_all_metricgroups.sh b/tools/perf/tests/shell/stat_all_metricgroups.sh
index cb35e488809a..eaa5e1172294 100755
--- a/tools/perf/tests/shell/stat_all_metricgroups.sh
+++ b/tools/perf/tests/shell/stat_all_metricgroups.sh
@@ -2,11 +2,19 @@
 # perf all metricgroups test
 # SPDX-License-Identifier: GPL-2.0
 
-set -e
-
 for m in $(perf list --raw-dump metricgroups); do
   echo "Testing $m"
-  perf stat -M "$m" -a true
+  result=$(perf stat -M "$m" -a true 2>&1)
+  rc=$?
+  # Skip if there is no access to perf_events monitoring
+  # Otherwise exit based on the return code of perf comamnd.
+  if echo "$result" | grep -q "Access to performance monitoring and observability operations is limited";
+  then
+      continue
+  else
+      [ $rc -ne 0 ] && exit $rc
+  fi
+
 done
 
 exit 0
-- 
2.31.1

Re: [PATCH V2 2/2] tools/perf/tests: perf all metricgroups test fails when perf_event access is restricted

[Disha Goel]

[-- Attachment #1: Type: text/plain, Size: 2419 bytes --]

On 04/08/23 10:30 am, Athira Rajeev wrote:
> Perf all metricgroups test fails as below when perf_event access
> is restricted.
>
>      ./perf test -v "perf all metricgroups test"
>      Testing Memory_BW
>      Error:
>      Access to performance monitoring and observability operations is limited.
>      Enforced MAC policy settings (SELinux) can limit access to performance
>      access to performance monitoring and observability operations for processes
>      without CAP_PERFMON, CAP_SYS_PTRACE or CAP_SYS_ADMIN Linux capability.
>
>      test child finished with -1
>      ---- end ----
>      perf all metricgroups test: FAILED!
>
> Fix the testcase to skip those metric events which needs perf_event access
> explicitly. The exit code of the testcase is based on return code of
> the perf stat command ( enabled by set -e option ). Hence save the
> exit status in a variable and use that to decide success or fail for the
> testcase.
>
> Signed-off-by: Athira Rajeev<atrajeev@linux.vnet.ibm.com>

With this patch applied(on power) perf metricgroups test works correctly when perf_event access is restricted.

  # ./perf test "perf all metricgroups test"
  96: perf all metricgroups test                                      : Ok

Tested-by: Disha Goel<disgoel@linux.ibm.com>

> ---
> Changelog:
> v1 -> v2:
>   Changed the condition to use "echo" and "grep" so it works on
>   Posix shell as well.
>
>   tools/perf/tests/shell/stat_all_metricgroups.sh | 14 +++++++++++---
>   1 file changed, 11 insertions(+), 3 deletions(-)
>
> diff --git a/tools/perf/tests/shell/stat_all_metricgroups.sh b/tools/perf/tests/shell/stat_all_metricgroups.sh
> index cb35e488809a..eaa5e1172294 100755
> --- a/tools/perf/tests/shell/stat_all_metricgroups.sh
> +++ b/tools/perf/tests/shell/stat_all_metricgroups.sh
> @@ -2,11 +2,19 @@
>   # perf all metricgroups test
>   # SPDX-License-Identifier: GPL-2.0
>
> -set -e
> -
>   for m in $(perf list --raw-dump metricgroups); do
>     echo "Testing $m"
> -  perf stat -M "$m" -a true
> +  result=$(perf stat -M "$m" -a true 2>&1)
> +  rc=$?
> +  # Skip if there is no access to perf_events monitoring
> +  # Otherwise exit based on the return code of perf comamnd.
> +  if echo "$result" | grep -q "Access to performance monitoring and observability operations is limited";
> +  then
> +      continue
> +  else
> +      [ $rc -ne 0 ] && exit $rc
> +  fi
> +
>   done
>
>   exit 0

[-- Attachment #2: Type: text/html, Size: 2981 bytes --]

Re: [PATCH V2 2/2] tools/perf/tests: perf all metricgroups test fails when perf_event access is restricted

[Arnaldo Carvalho de Melo]

Em Mon, Aug 07, 2023 at 08:14:39PM +0530, Disha Goel escreveu:
> On 04/08/23 10:30 am, Athira Rajeev wrote:
> > Perf all metricgroups test fails as below when perf_event access
> > is restricted.
> > 
> >      ./perf test -v "perf all metricgroups test"
> >      Testing Memory_BW
> >      Error:
> >      Access to performance monitoring and observability operations is limited.
> >      Enforced MAC policy settings (SELinux) can limit access to performance
> >      access to performance monitoring and observability operations for processes
> >      without CAP_PERFMON, CAP_SYS_PTRACE or CAP_SYS_ADMIN Linux capability.
> > 
> >      test child finished with -1
> >      ---- end ----
> >      perf all metricgroups test: FAILED!
> > 
> > Fix the testcase to skip those metric events which needs perf_event access
> > explicitly. The exit code of the testcase is based on return code of
> > the perf stat command ( enabled by set -e option ). Hence save the
> > exit status in a variable and use that to decide success or fail for the
> > testcase.

I wonder if we shouldn't somehow check if the credentials needed to
performing a test shouldn't be checked before trying it. This way we
would check if the check that the tool or the kernel is doing is the
appropriate one.

I.e. the kernel refusal for doing something may be an error.

- Arnaldo

> > Signed-off-by: Athira Rajeev<atrajeev@linux.vnet.ibm.com>
> 
> With this patch applied(on power) perf metricgroups test works correctly when perf_event access is restricted.
> 
>  # ./perf test "perf all metricgroups test"
>  96: perf all metricgroups test                                      : Ok
> 
> Tested-by: Disha Goel<disgoel@linux.ibm.com>
> 
> > ---
> > Changelog:
> > v1 -> v2:
> >   Changed the condition to use "echo" and "grep" so it works on
> >   Posix shell as well.
> > 
> >   tools/perf/tests/shell/stat_all_metricgroups.sh | 14 +++++++++++---
> >   1 file changed, 11 insertions(+), 3 deletions(-)
> > 
> > diff --git a/tools/perf/tests/shell/stat_all_metricgroups.sh b/tools/perf/tests/shell/stat_all_metricgroups.sh
> > index cb35e488809a..eaa5e1172294 100755
> > --- a/tools/perf/tests/shell/stat_all_metricgroups.sh
> > +++ b/tools/perf/tests/shell/stat_all_metricgroups.sh
> > @@ -2,11 +2,19 @@
> >   # perf all metricgroups test
> >   # SPDX-License-Identifier: GPL-2.0
> > 
> > -set -e
> > -
> >   for m in $(perf list --raw-dump metricgroups); do
> >     echo "Testing $m"
> > -  perf stat -M "$m" -a true
> > +  result=$(perf stat -M "$m" -a true 2>&1)
> > +  rc=$?
> > +  # Skip if there is no access to perf_events monitoring
> > +  # Otherwise exit based on the return code of perf comamnd.
> > +  if echo "$result" | grep -q "Access to performance monitoring and observability operations is limited";
> > +  then
> > +      continue
> > +  else
> > +      [ $rc -ne 0 ] && exit $rc
> > +  fi
> > +
> >   done
> > 
> >   exit 0

-- 

- Arnaldo

Re: [PATCH V2 2/2] tools/perf/tests: perf all metricgroups test fails when perf_event access is restricted

[Arnaldo Carvalho de Melo]

Em Mon, Aug 07, 2023 at 08:14:39PM +0530, Disha Goel escreveu:
> On 04/08/23 10:30 am, Athira Rajeev wrote:
> > Perf all metricgroups test fails as below when perf_event access
> > is restricted.
> > 
> >      ./perf test -v "perf all metricgroups test"
> >      Testing Memory_BW
> >      Error:
> >      Access to performance monitoring and observability operations is limited.
> >      Enforced MAC policy settings (SELinux) can limit access to performance
> >      access to performance monitoring and observability operations for processes
> >      without CAP_PERFMON, CAP_SYS_PTRACE or CAP_SYS_ADMIN Linux capability.
> > 
> >      test child finished with -1
> >      ---- end ----
> >      perf all metricgroups test: FAILED!
> > 
> > Fix the testcase to skip those metric events which needs perf_event access
> > explicitly. The exit code of the testcase is based on return code of
> > the perf stat command ( enabled by set -e option ). Hence save the
> > exit status in a variable and use that to decide success or fail for the
> > testcase.

I wonder if we shouldn't somehow check if the credentials needed to
performing a test shouldn't be checked before trying it. This way we
would check if the check that the tool or the kernel is doing is the
appropriate one.

I.e. the kernel refusal for doing something may be an error.

- Arnaldo

> > Signed-off-by: Athira Rajeev<atrajeev@linux.vnet.ibm.com>
> 
> With this patch applied(on power) perf metricgroups test works correctly when perf_event access is restricted.
> 
>  # ./perf test "perf all metricgroups test"
>  96: perf all metricgroups test                                      : Ok
> 
> Tested-by: Disha Goel<disgoel@linux.ibm.com>
> 
> > ---
> > Changelog:
> > v1 -> v2:
> >   Changed the condition to use "echo" and "grep" so it works on
> >   Posix shell as well.
> > 
> >   tools/perf/tests/shell/stat_all_metricgroups.sh | 14 +++++++++++---
> >   1 file changed, 11 insertions(+), 3 deletions(-)
> > 
> > diff --git a/tools/perf/tests/shell/stat_all_metricgroups.sh b/tools/perf/tests/shell/stat_all_metricgroups.sh
> > index cb35e488809a..eaa5e1172294 100755
> > --- a/tools/perf/tests/shell/stat_all_metricgroups.sh
> > +++ b/tools/perf/tests/shell/stat_all_metricgroups.sh
> > @@ -2,11 +2,19 @@
> >   # perf all metricgroups test
> >   # SPDX-License-Identifier: GPL-2.0
> > 
> > -set -e
> > -
> >   for m in $(perf list --raw-dump metricgroups); do
> >     echo "Testing $m"
> > -  perf stat -M "$m" -a true
> > +  result=$(perf stat -M "$m" -a true 2>&1)
> > +  rc=$?
> > +  # Skip if there is no access to perf_events monitoring
> > +  # Otherwise exit based on the return code of perf comamnd.
> > +  if echo "$result" | grep -q "Access to performance monitoring and observability operations is limited";
> > +  then
> > +      continue
> > +  else
> > +      [ $rc -ne 0 ] && exit $rc
> > +  fi
> > +
> >   done
> > 
> >   exit 0

-- 

- Arnaldo