Re: [PATCH 2/2] perf test: Add basic stat and topdown group test

["Liang, Kan" <kan.liang@linux.intel.com>]

On 5/5/2022 2:35 PM, Ian Rogers wrote:
> On Thu, May 5, 2022 at 11:19 AM Liang, Kan <kan.liang@linux.intel.com> wrote:
>>
>> On 5/5/2022 11:22 AM, Ian Rogers wrote:
>>> On Thu, May 5, 2022 at 5:12 AM Liang, Kan <kan.liang@linux.intel.com> wrote:
>>>>
>>>> On 5/5/2022 12:38 AM, Ian Rogers wrote:
>>>>> Add a basic stat test.
>>>>> Add two tests of grouping behavior for topdown events. Topdown events
>>>>> are special as they must be grouped with the slots event first.
>>>>>
>>>>> Signed-off-by: Ian Rogers <irogers@google.com>
>>>>> ---
>>>>>     tools/perf/tests/shell/stat.sh | 65 ++++++++++++++++++++++++++++++++++
>>>>>     1 file changed, 65 insertions(+)
>>>>>     create mode 100755 tools/perf/tests/shell/stat.sh
>>>>>
>>>>> diff --git a/tools/perf/tests/shell/stat.sh b/tools/perf/tests/shell/stat.sh
>>>>> new file mode 100755
>>>>> index 000000000000..80869ea6debc
>>>>> --- /dev/null
>>>>> +++ b/tools/perf/tests/shell/stat.sh
>>>>> @@ -0,0 +1,65 @@
>>>>> +#!/bin/sh
>>>>> +# perf stat tests
>>>>> +# SPDX-License-Identifier: GPL-2.0
>>>>> +
>>>>> +set -e
>>>>> +
>>>>> +err=0
>>>>> +test_default_stat() {
>>>>> +  echo "Basic stat command test"
>>>>> +  if ! perf stat true 2>&1 | egrep -q "Performance counter stats for 'true':"
>>>>> +  then
>>>>> +    echo "Basic stat command test [Failed]"
>>>>> +    err=1
>>>>> +    return
>>>>> +  fi
>>>>> +  echo "Basic stat command test [Success]"
>>>>> +}
>>>>> +
>>>>> +test_topdown_groups() {
>>>>> +  # Topdown events must be grouped with the slots event first. Test that
>>>>> +  # parse-events reorders this.
>>>>> +  echo "Topdown event group test"
>>>>> +  if ! perf stat -e '{slots,topdown-retiring}' true > /dev/null 2>&1
>>>>> +  then
>>>>> +    echo "Topdown event group test [Skipped event parsing failed]"
>>>>> +    return
>>>>> +  fi
>>>>> +  if perf stat -e '{slots,topdown-retiring}' true 2>&1 | egrep -q "<not supported>"
>>>>> +  then
>>>>> +    echo "Topdown event group test [Failed events not supported]"
>>>>> +    err=1
>>>>> +    return
>>>>> +  fi
>>>>> +  if perf stat -e '{topdown-retiring,slots}' true 2>&1 | egrep -q "<not supported>"
>>>>> +  then
>>>>> +    echo "Topdown event group test [Failed slots not reordered first]"
>>>>> +    err=1
>>>>> +    return
>>>>> +  fi
>>>>> +  echo "Topdown event group test [Success]"
>>>>> +}
>>>>> +
>>>>> +test_topdown_weak_groups() {
>>>>> +  # Weak groups break if the perf_event_open of multiple grouped events
>>>>> +  # fails. Breaking a topdown group causes the events to fail. Test a very large
>>>>> +  # grouping to see that the topdown events aren't broken out.
>>>>> +  echo "Topdown weak groups test"
>>>>> +  if ! perf stat -e '{slots,topdown-bad-spec,topdown-be-bound,topdown-fe-bound,topdown-retiring},branch-instructions,branch-misses,bus-cycles,cache-misses,cache-references,cpu-cycles,instructions,mem-loads,mem-stores,ref-cycles,baclears.any,ARITH.DIVIDER_ACTIVE' true > /dev/null 2>&1
>>>>> +  then
>>>>> +    echo "Topdown weak groups test [Skipped event parsing failed]"
>>>>> +    return
>>>>> +  fi
>>>>> +  if perf stat -e '{slots,topdown-bad-spec,topdown-be-bound,topdown-fe-bound,topdown-retiring,branch-instructions,branch-misses,bus-cycles,cache-misses,cache-references,cpu-cycles,instructions,mem-loads,mem-stores,ref-cycles,baclears.any,ARITH.DIVIDER_ACTIVE}:W' true 2>&1 | egrep -q "<not supported>"
>>>>> +  then
>>>>> +    echo "Topdown weak groups test [Failed events not supported]"
>>>>> +    err=1
>>>>> +    return
>>>>> +  fi
>>>>> +  echo "Topdown weak groups test [Success]"
>>>>> +}
>>>>> +
>>>>
>>>> Should we check the existence of the slots event before the test?
>>>> The perf metrics feature only be available on the new platform after
>>>> ICL. It doesn't work on Atom.
>>>>
>>>> Also, I think the test may fails on the hybrid platform, since big core
>>>> and small core have different formula for the topdown. I think we should
>>>> avoid the test for the hybrid platform for now.
>>>> +Zhengjun, who is fixing the topdown gap for the hybrid platform. I
>>>> think he may take care of the hybrid support later.
>>>
>>> Thanks Kan, the test filters out systems that don't support the events
>>> and silently skips the test. The main purpose of the test is to make
>>> sure the somewhat complicated grouping operations for Icelake have
>>> some coverage. Adding more coverage for hybrid would be great, but not
>>> something I think gates this change.
>>>
>>
>> Sure, we can add the coverage for hybrid later. But please make sure the
>> test can filter out both the systems which doesn't support perf metircs
>> and the hybrid system.
>>
>> Thanks,
>> Kan
> 
> If the test fails on hybrid then that feels like value add :-) 

Indeed. The test case is valid.

Could you please use architecture events to replace the 
baclears.any,ARITH.DIVIDER_ACTIVE?
It's not guaranteed that the two events are supported for the future 
platforms.

I use the duplicated cache-misses,cache-references events to replace 
baclears.any,ARITH.DIVIDER_ACTIVE and tested on the ADL. The results 
look good.

./perf stat -e 
'{slots,topdown-bad-spec,topdown-be-bound,topdown-fe-bound,topdown-retiring,branch-instructions,branch-misses,bus-cycles,cache-misses,cache-references,cpu-cycles,instructions,mem-loads,mem-stores,ref-cycles,cache-misses,cache-references}:W' 
true
WARNING: events in group from different hybrid PMUs!

  Performance counter stats for 'true':

          3,603,798      cpu_core/slots/
            480,506      cpu_core/topdown-bad-spec/ #     13.3% bad 
speculation
      <not counted>      cpu_atom/topdown-bad-spec/ 
                 (0.00%)
            833,819      cpu_core/topdown-be-bound/ #     23.0% backend 
bound
      <not counted>      cpu_atom/topdown-be-bound/ 
                 (0.00%)
          1,483,916      cpu_core/topdown-fe-bound/ #     41.0% frontend 
bound
      <not counted>      cpu_atom/topdown-fe-bound/ 
                 (0.00%)
            819,687      cpu_core/topdown-retiring/ #     22.7% retiring
      <not counted>      cpu_atom/topdown-retiring/ 
                 (0.00%)
            151,010      cpu_core/branch-instructions/
      <not counted>      cpu_atom/branch-instructions/ 
                    (0.00%)
              4,402      cpu_core/branch-misses/
      <not counted>      cpu_atom/branch-misses/ 
                (0.00%)
          1,456,308      cpu_core/bus-cycles/
      <not counted>      cpu_atom/bus-cycles/ 
                (0.00%)
              1,132      cpu_core/cache-misses/
      <not counted>      cpu_atom/cache-misses/ 
                (0.00%)
             14,226      cpu_core/cache-references/
      <not counted>      cpu_atom/cache-references/ 
                 (0.00%)
            600,633      cpu_core/cpu-cycles/
      <not counted>      cpu_atom/cpu-cycles/ 
                (0.00%)
            737,806      cpu_core/instructions/
      <not counted>      cpu_atom/instructions/ 
                (0.00%)
                  0      cpu_core/mem-loads/
      <not counted>      cpu_atom/mem-loads/ 
                (0.00%)
            120,151      cpu_core/mem-stores/
      <not counted>      cpu_atom/mem-stores/ 
                (0.00%)
          1,456,308      cpu_core/ref-cycles/
      <not counted>      cpu_atom/ref-cycles/ 
                (0.00%)
      <not counted>      cpu_core/cache-misses/ 
                (0.00%)
      <not counted>      cpu_atom/cache-misses/ 
                (0.00%)
      <not counted>      cpu_core/cache-references/ 
                 (0.00%)
      <not counted>      cpu_atom/cache-references/ 
                 (0.00%)

        0.001518955 seconds time elapsed

        0.001531000 seconds user
        0.000000000 seconds sys

Thanks,
Kan
> We
> genuinely have broken grouping functions. We could just add to the
> test a skip if /sys/devices/cpu_core and /sys/devices/cpu_atom
> directories exist (making assumptions on where sysfs is mounted). I'm
> not yet able to test on Alderlake hence not wanting to have a lot of
> untested code.
> 
> Thanks,
> Ian
> 
>>> Thanks,
>>> Ian
>>>
>>>> Thanks,
>>>> Kan
>>>>> +test_default_stat
>>>>> +test_topdown_groups
>>>>> +test_topdown_weak_groups
>>>>> +exit $err