[PATCH 1/2] tools/perf/tests: perf all metrics test fails when perf_event access is restricted

[PATCH 1/2] tools/perf/tests: perf all metrics test fails when perf_event access is restricted

[Athira Rajeev]

Perf all metrics test fails as below when perf_event access
is restricted.

    ./perf test -v "perf all metrics test"
    Metric 'Memory_RD_BW_Chip' not printed in:
    Error:
    Access to performance monitoring and observability operations is limited.
    Enforced MAC policy settings (SELinux) can limit access to performance
    —
    access to performance monitoring and observability operations for processes
    without CAP_PERFMON, CAP_SYS_PTRACE or CAP_SYS_ADMIN Linux capability.
    —
    test child finished with -1
    ---- end ----
    perf all metrics test: FAILED!

The perf all metrics test picks the input events from
"perf list --raw-dump metrics" and runs "perf stat -M "$m""
for each of the metrics in the list. It fails here for some
of the metrics which needs access, since it collects system
wide resource details/statistics. Fix the testcase to skip
those metric events.

Signed-off-by: Athira Rajeev <atrajeev@linux.vnet.ibm.com>
---
 tools/perf/tests/shell/stat_all_metrics.sh | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/tools/perf/tests/shell/stat_all_metrics.sh b/tools/perf/tests/shell/stat_all_metrics.sh
index 54774525e18a..14b96484a359 100755
--- a/tools/perf/tests/shell/stat_all_metrics.sh
+++ b/tools/perf/tests/shell/stat_all_metrics.sh
@@ -6,7 +6,9 @@ err=0
 for m in $(perf list --raw-dump metrics); do
   echo "Testing $m"
   result=$(perf stat -M "$m" true 2>&1)
-  if [[ "$result" =~ ${m:0:50} ]] || [[ "$result" =~ "<not supported>" ]]
+  # Skip if there is no access to perf_events monitoring
+  # and observability operations
+  if [[ "$result" =~ ${m:0:50} ]] || [[ "$result" =~ "<not supported>" ]] || [[ "$result" =~ "Access to performance monitoring and observability operations is limited" ]]
   then
     continue
   fi
-- 
2.31.1

[PATCH 2/2] tools/perf/tests: perf all metricgroups test fails when perf_event access is restricted

[Athira Rajeev]

Perf all metricgroups test fails as below when perf_event access
is restricted.

    ./perf test -v "perf all metricgroups test"
    Testing Memory_BW
    Error:
    Access to performance monitoring and observability operations is limited.
    Enforced MAC policy settings (SELinux) can limit access to performance
    —
    access to performance monitoring and observability operations for processes
    without CAP_PERFMON, CAP_SYS_PTRACE or CAP_SYS_ADMIN Linux capability.
    —
    test child finished with -1
    ---- end ----
    perf all metricgroups test: FAILED!

Fix the testcase to skip those metric events which needs perf_event access
explicitly. The exit code of the testcase is based on return code of
the perf stat command ( enabled by set -e option ). Hence save the
exit status in a variable and use that to decide success or fail for the
testcase.

Signed-off-by: Athira Rajeev <atrajeev@linux.vnet.ibm.com>
---
 tools/perf/tests/shell/stat_all_metricgroups.sh | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)

diff --git a/tools/perf/tests/shell/stat_all_metricgroups.sh b/tools/perf/tests/shell/stat_all_metricgroups.sh
index cb35e488809a..fa86b562676a 100755
--- a/tools/perf/tests/shell/stat_all_metricgroups.sh
+++ b/tools/perf/tests/shell/stat_all_metricgroups.sh
@@ -2,11 +2,19 @@
 # perf all metricgroups test
 # SPDX-License-Identifier: GPL-2.0
 
-set -e
-
 for m in $(perf list --raw-dump metricgroups); do
   echo "Testing $m"
-  perf stat -M "$m" -a true
+  result=$(perf stat -M "$m" -a true 2>&1)
+  rc=$?
+  # Skip if there is no access to perf_events monitoring
+  # Otherwise exit based on the return code of perf comamnd.
+  if [[ "$result" =~ "Access to performance monitoring and observability operations is limited" ]]
+  then
+      continue
+  else
+      [ $rc -ne 0 ] && exit $rc
+  fi
+
 done
 
 exit 0
-- 
2.31.1

Re: [PATCH 2/2] tools/perf/tests: perf all metricgroups test fails when perf_event access is restricted

[Athira Rajeev]

> On 15-Jun-2023, at 1:08 PM, Athira Rajeev <atrajeev@linux.vnet.ibm.com> wrote:
> 
> Perf all metricgroups test fails as below when perf_event access
> is restricted.
> 
>    ./perf test -v "perf all metricgroups test"
>    Testing Memory_BW
>    Error:
>    Access to performance monitoring and observability operations is limited.
>    Enforced MAC policy settings (SELinux) can limit access to performance
>    —
>    access to performance monitoring and observability operations for processes
>    without CAP_PERFMON, CAP_SYS_PTRACE or CAP_SYS_ADMIN Linux capability.
>    —
>    test child finished with -1
>    ---- end ----
>    perf all metricgroups test: FAILED!
> 
> Fix the testcase to skip those metric events which needs perf_event access
> explicitly. The exit code of the testcase is based on return code of
> the perf stat command ( enabled by set -e option ). Hence save the
> exit status in a variable and use that to decide success or fail for the
> testcase.
> 
> Signed-off-by: Athira Rajeev <atrajeev@linux.vnet.ibm.com>
> ---
> tools/perf/tests/shell/stat_all_metricgroups.sh | 14 +++++++++++---
> 1 file changed, 11 insertions(+), 3 deletions(-)
> 
> diff --git a/tools/perf/tests/shell/stat_all_metricgroups.sh b/tools/perf/tests/shell/stat_all_metricgroups.sh
> index cb35e488809a..fa86b562676a 100755
> --- a/tools/perf/tests/shell/stat_all_metricgroups.sh
> +++ b/tools/perf/tests/shell/stat_all_metricgroups.sh
> @@ -2,11 +2,19 @@
> # perf all metricgroups test
> # SPDX-License-Identifier: GPL-2.0
> 
> -set -e
> -
> for m in $(perf list --raw-dump metricgroups); do
>   echo "Testing $m"
> -  perf stat -M "$m" -a true
> +  result=$(perf stat -M "$m" -a true 2>&1)
> +  rc=$?
> +  # Skip if there is no access to perf_events monitoring
> +  # Otherwise exit based on the return code of perf comamnd.
> +  if [[ "$result" =~ "Access to performance monitoring and observability operations is limited" ]]
> +  then
> +      continue
> +  else
> +      [ $rc -ne 0 ] && exit $rc
> +  fi
> +
> done

Hi,

Looking for review comments on this patch.

Thanks
Athira
> 
> exit 0
> -- 
> 2.31.1
> 

Re: [PATCH 1/2] tools/perf/tests: perf all metrics test fails when perf_event access is restricted

[Athira Rajeev]

> On 15-Jun-2023, at 1:08 PM, Athira Rajeev <atrajeev@linux.vnet.ibm.com> wrote:
> 
> Perf all metrics test fails as below when perf_event access
> is restricted.
> 
>    ./perf test -v "perf all metrics test"
>    Metric 'Memory_RD_BW_Chip' not printed in:
>    Error:
>    Access to performance monitoring and observability operations is limited.
>    Enforced MAC policy settings (SELinux) can limit access to performance
>    —
>    access to performance monitoring and observability operations for processes
>    without CAP_PERFMON, CAP_SYS_PTRACE or CAP_SYS_ADMIN Linux capability.
>    —
>    test child finished with -1
>    ---- end ----
>    perf all metrics test: FAILED!


Hi,

Looking for review comments on this patch.

Thanks
> 
> The perf all metrics test picks the input events from
> "perf list --raw-dump metrics" and runs "perf stat -M "$m""
> for each of the metrics in the list. It fails here for some
> of the metrics which needs access, since it collects system
> wide resource details/statistics. Fix the testcase to skip
> those metric events.
> 
> Signed-off-by: Athira Rajeev <atrajeev@linux.vnet.ibm.com>
> ---
> tools/perf/tests/shell/stat_all_metrics.sh | 4 +++-
> 1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/tools/perf/tests/shell/stat_all_metrics.sh b/tools/perf/tests/shell/stat_all_metrics.sh
> index 54774525e18a..14b96484a359 100755
> --- a/tools/perf/tests/shell/stat_all_metrics.sh
> +++ b/tools/perf/tests/shell/stat_all_metrics.sh
> @@ -6,7 +6,9 @@ err=0
> for m in $(perf list --raw-dump metrics); do
>   echo "Testing $m"
>   result=$(perf stat -M "$m" true 2>&1)
> -  if [[ "$result" =~ ${m:0:50} ]] || [[ "$result" =~ "<not supported>" ]]
> +  # Skip if there is no access to perf_events monitoring
> +  # and observability operations
> +  if [[ "$result" =~ ${m:0:50} ]] || [[ "$result" =~ "<not supported>" ]] || [[ "$result" =~ "Access to performance monitoring and observability operations is limited" ]]
>   then
>     continue
>   fi
> -- 
> 2.31.1
> 

Re: [PATCH 1/2] tools/perf/tests: perf all metrics test fails when perf_event access is restricted

[Namhyung Kim]

On Mon, Jul 3, 2023 at 10:04 PM Athira Rajeev
<atrajeev@linux.vnet.ibm.com> wrote:
>
>
>
> > On 15-Jun-2023, at 1:08 PM, Athira Rajeev <atrajeev@linux.vnet.ibm.com> wrote:
> >
> > Perf all metrics test fails as below when perf_event access
> > is restricted.
> >
> >    ./perf test -v "perf all metrics test"
> >    Metric 'Memory_RD_BW_Chip' not printed in:
> >    Error:
> >    Access to performance monitoring and observability operations is limited.
> >    Enforced MAC policy settings (SELinux) can limit access to performance
> >    —
> >    access to performance monitoring and observability operations for processes
> >    without CAP_PERFMON, CAP_SYS_PTRACE or CAP_SYS_ADMIN Linux capability.
> >    —
> >    test child finished with -1
> >    ---- end ----
> >    perf all metrics test: FAILED!

In my system, it fails like below:

  $ ./perf test -v 101
  101: perf all metrics test                                           :
  --- start ---
  test child forked, pid 398458
  Testing branch_misprediction_ratio
  Testing all_remote_links_outbound
  Metric 'all_remote_links_outbound' not printed in:
  Error:
  Invalid event (remote_outbound_data_controller_3:u) in per-thread
mode, enable system wide with '-a'.
  Testing nps1_die_to_dram
  ...

Thanks,
Namhyung

>
>
> Hi,
>
> Looking for review comments on this patch.
>
> Thanks
> >
> > The perf all metrics test picks the input events from
> > "perf list --raw-dump metrics" and runs "perf stat -M "$m""
> > for each of the metrics in the list. It fails here for some
> > of the metrics which needs access, since it collects system
> > wide resource details/statistics. Fix the testcase to skip
> > those metric events.
> >
> > Signed-off-by: Athira Rajeev <atrajeev@linux.vnet.ibm.com>
> > ---
> > tools/perf/tests/shell/stat_all_metrics.sh | 4 +++-
> > 1 file changed, 3 insertions(+), 1 deletion(-)
> >
> > diff --git a/tools/perf/tests/shell/stat_all_metrics.sh b/tools/perf/tests/shell/stat_all_metrics.sh
> > index 54774525e18a..14b96484a359 100755
> > --- a/tools/perf/tests/shell/stat_all_metrics.sh
> > +++ b/tools/perf/tests/shell/stat_all_metrics.sh
> > @@ -6,7 +6,9 @@ err=0
> > for m in $(perf list --raw-dump metrics); do
> >   echo "Testing $m"
> >   result=$(perf stat -M "$m" true 2>&1)
> > -  if [[ "$result" =~ ${m:0:50} ]] || [[ "$result" =~ "<not supported>" ]]
> > +  # Skip if there is no access to perf_events monitoring
> > +  # and observability operations
> > +  if [[ "$result" =~ ${m:0:50} ]] || [[ "$result" =~ "<not supported>" ]] || [[ "$result" =~ "Access to performance monitoring and observability operations is limited" ]]
> >   then
> >     continue
> >   fi
> > --
> > 2.31.1
> >
>

Re: [PATCH 1/2] tools/perf/tests: perf all metrics test fails when perf_event access is restricted

[Athira Rajeev]

> On 05-Jul-2023, at 11:54 PM, Namhyung Kim <namhyung@kernel.org> wrote:
> 
> On Mon, Jul 3, 2023 at 10:04 PM Athira Rajeev
> <atrajeev@linux.vnet.ibm.com> wrote:
>> 
>> 
>> 
>>> On 15-Jun-2023, at 1:08 PM, Athira Rajeev <atrajeev@linux.vnet.ibm.com> wrote:
>>> 
>>> Perf all metrics test fails as below when perf_event access
>>> is restricted.
>>> 
>>>   ./perf test -v "perf all metrics test"
>>>   Metric 'Memory_RD_BW_Chip' not printed in:
>>>   Error:
>>>   Access to performance monitoring and observability operations is limited.
>>>   Enforced MAC policy settings (SELinux) can limit access to performance
>>>   —
>>>   access to performance monitoring and observability operations for processes
>>>   without CAP_PERFMON, CAP_SYS_PTRACE or CAP_SYS_ADMIN Linux capability.
>>>   —
>>>   test child finished with -1
>>>   ---- end ----
>>>   perf all metrics test: FAILED!
> 
> In my system, it fails like below:
> 
>  $ ./perf test -v 101
>  101: perf all metrics test                                           :
>  --- start ---
>  test child forked, pid 398458
>  Testing branch_misprediction_ratio
>  Testing all_remote_links_outbound
>  Metric 'all_remote_links_outbound' not printed in:
>  Error:
>  Invalid event (remote_outbound_data_controller_3:u) in per-thread
> mode, enable system wide with '-a'.
>  Testing nps1_die_to_dram

Hi Namhyung,

Thanks for checking. Is the fail behaviour observed after applying this fix patch ?
I will check it in my setup 

Athira
>  ...
> 
> Thanks,
> Namhyung
> 
>> 
>> 
>> Hi,
>> 
>> Looking for review comments on this patch.
>> 
>> Thanks
>>> 
>>> The perf all metrics test picks the input events from
>>> "perf list --raw-dump metrics" and runs "perf stat -M "$m""
>>> for each of the metrics in the list. It fails here for some
>>> of the metrics which needs access, since it collects system
>>> wide resource details/statistics. Fix the testcase to skip
>>> those metric events.
>>> 
>>> Signed-off-by: Athira Rajeev <atrajeev@linux.vnet.ibm.com>
>>> ---
>>> tools/perf/tests/shell/stat_all_metrics.sh | 4 +++-
>>> 1 file changed, 3 insertions(+), 1 deletion(-)
>>> 
>>> diff --git a/tools/perf/tests/shell/stat_all_metrics.sh b/tools/perf/tests/shell/stat_all_metrics.sh
>>> index 54774525e18a..14b96484a359 100755
>>> --- a/tools/perf/tests/shell/stat_all_metrics.sh
>>> +++ b/tools/perf/tests/shell/stat_all_metrics.sh
>>> @@ -6,7 +6,9 @@ err=0
>>> for m in $(perf list --raw-dump metrics); do
>>>  echo "Testing $m"
>>>  result=$(perf stat -M "$m" true 2>&1)
>>> -  if [[ "$result" =~ ${m:0:50} ]] || [[ "$result" =~ "<not supported>" ]]
>>> +  # Skip if there is no access to perf_events monitoring
>>> +  # and observability operations
>>> +  if [[ "$result" =~ ${m:0:50} ]] || [[ "$result" =~ "<not supported>" ]] || [[ "$result" =~ "Access to performance monitoring and observability operations is limited" ]]
>>>  then
>>>    continue
>>>  fi
>>> --
>>> 2.31.1

Re: [PATCH 1/2] tools/perf/tests: perf all metrics test fails when perf_event access is restricted

[Athira Rajeev]

> On 05-Jul-2023, at 11:54 PM, Namhyung Kim <namhyung@kernel.org> wrote:
> 
> On Mon, Jul 3, 2023 at 10:04 PM Athira Rajeev
> <atrajeev@linux.vnet.ibm.com> wrote:
>> 
>> 
>> 
>>> On 15-Jun-2023, at 1:08 PM, Athira Rajeev <atrajeev@linux.vnet.ibm.com> wrote:
>>> 
>>> Perf all metrics test fails as below when perf_event access
>>> is restricted.
>>> 
>>>   ./perf test -v "perf all metrics test"
>>>   Metric 'Memory_RD_BW_Chip' not printed in:
>>>   Error:
>>>   Access to performance monitoring and observability operations is limited.
>>>   Enforced MAC policy settings (SELinux) can limit access to performance
>>>   —
>>>   access to performance monitoring and observability operations for processes
>>>   without CAP_PERFMON, CAP_SYS_PTRACE or CAP_SYS_ADMIN Linux capability.
>>>   —
>>>   test child finished with -1
>>>   ---- end ----
>>>   perf all metrics test: FAILED!
> 
> In my system, it fails like below:
> 
>  $ ./perf test -v 101
>  101: perf all metrics test                                           :
>  --- start ---
>  test child forked, pid 398458
>  Testing branch_misprediction_ratio
>  Testing all_remote_links_outbound
>  Metric 'all_remote_links_outbound' not printed in:
>  Error:
>  Invalid event (remote_outbound_data_controller_3:u) in per-thread
> mode, enable system wide with '-a'.
>  Testing nps1_die_to_dram
>  ...
> 
> Thanks,
> Namhyung



Hi Namhyung,

I have posted a V2 for this :
[PATCH V2 1/2] tools/perf/tests: perf all metrics test fails when perf_event access is restricted
[PATCH V2 2/2] tools/perf/tests: perf all metricgroups test fails when perf_event access is restricted

Please review and test the V2 version, and let me know your feedback.

If any other metric fails even with this fix, that would be an event fail I believe and not a test issue.
But looking for your test results with the V2 patchset

Thanks
Athira
> 
>> 
>> 
>> Hi,
>> 
>> Looking for review comments on this patch.
>> 
>> Thanks
>>> 
>>> The perf all metrics test picks the input events from
>>> "perf list --raw-dump metrics" and runs "perf stat -M "$m""
>>> for each of the metrics in the list. It fails here for some
>>> of the metrics which needs access, since it collects system
>>> wide resource details/statistics. Fix the testcase to skip
>>> those metric events.
>>> 
>>> Signed-off-by: Athira Rajeev <atrajeev@linux.vnet.ibm.com>
>>> ---
>>> tools/perf/tests/shell/stat_all_metrics.sh | 4 +++-
>>> 1 file changed, 3 insertions(+), 1 deletion(-)
>>> 
>>> diff --git a/tools/perf/tests/shell/stat_all_metrics.sh b/tools/perf/tests/shell/stat_all_metrics.sh
>>> index 54774525e18a..14b96484a359 100755
>>> --- a/tools/perf/tests/shell/stat_all_metrics.sh
>>> +++ b/tools/perf/tests/shell/stat_all_metrics.sh
>>> @@ -6,7 +6,9 @@ err=0
>>> for m in $(perf list --raw-dump metrics); do
>>>  echo "Testing $m"
>>>  result=$(perf stat -M "$m" true 2>&1)
>>> -  if [[ "$result" =~ ${m:0:50} ]] || [[ "$result" =~ "<not supported>" ]]
>>> +  # Skip if there is no access to perf_events monitoring
>>> +  # and observability operations
>>> +  if [[ "$result" =~ ${m:0:50} ]] || [[ "$result" =~ "<not supported>" ]] || [[ "$result" =~ "Access to performance monitoring and observability operations is limited" ]]
>>>  then
>>>    continue
>>>  fi
>>> --
>>> 2.31.1