[PATCH v1] perf test stat: Avoid hybrid assumption when virtualized

[PATCH v1] perf test stat: Avoid hybrid assumption when virtualized

[Ian Rogers]

The cycles event will fallback to task-clock in the hybrid test when
running virtualized. Change the test to not fail for this.

Fixes: a6b8bb2addd0 ("perf test: Add a test for default perf stat command")
Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/tests/shell/stat.sh | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/tools/perf/tests/shell/stat.sh b/tools/perf/tests/shell/stat.sh
index 5a2ca2bcf94d..60cea07350e1 100755
--- a/tools/perf/tests/shell/stat.sh
+++ b/tools/perf/tests/shell/stat.sh
@@ -165,9 +165,13 @@ test_hybrid() {
 
   if [ "$pmus" -ne "$cycles_events" ]
   then
-    echo "hybrid test [Found $pmus PMUs but $cycles_events cycles events. Failed]"
-    err=1
-    return
+     # If virtualized the software task-clock event will be used.
+     if ! perf stat -- true 2>&1 | grep -q "task-clock"
+     then
+       echo "hybrid test [Found $pmus PMUs but $cycles_events cycles events. Failed]"
+       err=1
+       return
+     fi
   fi
   echo "hybrid test [Success]"
 }
-- 
2.47.1.613.gc27f4b7a9f-goog

Re: [PATCH v1] perf test stat: Avoid hybrid assumption when virtualized

[James Clark]

On 11/12/2024 6:10 am, Ian Rogers wrote:
> The cycles event will fallback to task-clock in the hybrid test when
> running virtualized. Change the test to not fail for this.
> 
> Fixes: a6b8bb2addd0 ("perf test: Add a test for default perf stat command")
> Signed-off-by: Ian Rogers <irogers@google.com>
> ---
>   tools/perf/tests/shell/stat.sh | 10 +++++++---
>   1 file changed, 7 insertions(+), 3 deletions(-)
> 
> diff --git a/tools/perf/tests/shell/stat.sh b/tools/perf/tests/shell/stat.sh
> index 5a2ca2bcf94d..60cea07350e1 100755
> --- a/tools/perf/tests/shell/stat.sh
> +++ b/tools/perf/tests/shell/stat.sh
> @@ -165,9 +165,13 @@ test_hybrid() {
>   
>     if [ "$pmus" -ne "$cycles_events" ]
>     then
> -    echo "hybrid test [Found $pmus PMUs but $cycles_events cycles events. Failed]"
> -    err=1
> -    return
> +     # If virtualized the software task-clock event will be used.
> +     if ! perf stat -- true 2>&1 | grep -q "task-clock"
> +     then
> +       echo "hybrid test [Found $pmus PMUs but $cycles_events cycles events. Failed]"
> +       err=1
> +       return
> +     fi
>     fi
>     echo "hybrid test [Success]"
>   }

Hi Ian,

Isn't the distinction between task-clock and cpu-clock whether the event 
is per-cpu or not?

$ perf stat -C 1 -- true 2>&1 | grep cpu-clock
               1.49 msec cpu-clock       #    0.917 CPUs utilized

$ perf stat -- true 2>&1 | grep task-clock
               0.30 msec task-clock      #    0.366 CPUs utilized

The test uses per-task mode so this change makes it always pass, even 
when the number of cycles events doesn't match the PMUs.

Thanks
James

Re: [PATCH v1] perf test stat: Avoid hybrid assumption when virtualized

[Ian Rogers]

On Wed, Dec 11, 2024 at 1:50 AM James Clark <james.clark@linaro.org> wrote:
>
> On 11/12/2024 6:10 am, Ian Rogers wrote:
> > The cycles event will fallback to task-clock in the hybrid test when
> > running virtualized. Change the test to not fail for this.
> >
> > Fixes: a6b8bb2addd0 ("perf test: Add a test for default perf stat command")
> > Signed-off-by: Ian Rogers <irogers@google.com>
> > ---
> >   tools/perf/tests/shell/stat.sh | 10 +++++++---
> >   1 file changed, 7 insertions(+), 3 deletions(-)
> >
> > diff --git a/tools/perf/tests/shell/stat.sh b/tools/perf/tests/shell/stat.sh
> > index 5a2ca2bcf94d..60cea07350e1 100755
> > --- a/tools/perf/tests/shell/stat.sh
> > +++ b/tools/perf/tests/shell/stat.sh
> > @@ -165,9 +165,13 @@ test_hybrid() {
> >
> >     if [ "$pmus" -ne "$cycles_events" ]
> >     then
> > -    echo "hybrid test [Found $pmus PMUs but $cycles_events cycles events. Failed]"
> > -    err=1
> > -    return
> > +     # If virtualized the software task-clock event will be used.
> > +     if ! perf stat -- true 2>&1 | grep -q "task-clock"
> > +     then
> > +       echo "hybrid test [Found $pmus PMUs but $cycles_events cycles events. Failed]"
> > +       err=1
> > +       return
> > +     fi
> >     fi
> >     echo "hybrid test [Success]"
> >   }
>
> Hi Ian,
>
> Isn't the distinction between task-clock and cpu-clock whether the event
> is per-cpu or not?
>
> $ perf stat -C 1 -- true 2>&1 | grep cpu-clock
>                1.49 msec cpu-clock       #    0.917 CPUs utilized
>
> $ perf stat -- true 2>&1 | grep task-clock
>                0.30 msec task-clock      #    0.366 CPUs utilized
>
> The test uses per-task mode so this change makes it always pass, even
> when the number of cycles events doesn't match the PMUs.

So I'm confused by the test, but it has caused a passing test to be
broken for me when I run virtualized. The test is checking a cycles
event is opened on each hybrid PMU, but this is conflated with
checking perf stat's "default" output. The cycles event will fall back
to task-clock in per-task mode but we also open a task-clock in
default mode. Should:
```
if [ "$pmus" -ne "$cycles_events" ]
```
be something like this then:
```
# The expectation is that default output will have a cycles events on
each hybrid
# PMU, but in situations with no cycles PMU events, like
virtualized,this can fall
# back to task-clock and so the end count may be 0. Fail if neither
condition holds.
if [ "$pmus" -ne "$cycles_events" ] && [ "$pmus" -ne "0" ]
```

Thanks,
Ian

Re: [PATCH v1] perf test stat: Avoid hybrid assumption when virtualized

[James Clark]

On 11/12/2024 5:58 pm, Ian Rogers wrote:
> On Wed, Dec 11, 2024 at 1:50 AM James Clark <james.clark@linaro.org> wrote:
>>
>> On 11/12/2024 6:10 am, Ian Rogers wrote:
>>> The cycles event will fallback to task-clock in the hybrid test when
>>> running virtualized. Change the test to not fail for this.
>>>
>>> Fixes: a6b8bb2addd0 ("perf test: Add a test for default perf stat command")
>>> Signed-off-by: Ian Rogers <irogers@google.com>
>>> ---
>>>    tools/perf/tests/shell/stat.sh | 10 +++++++---
>>>    1 file changed, 7 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/tools/perf/tests/shell/stat.sh b/tools/perf/tests/shell/stat.sh
>>> index 5a2ca2bcf94d..60cea07350e1 100755
>>> --- a/tools/perf/tests/shell/stat.sh
>>> +++ b/tools/perf/tests/shell/stat.sh
>>> @@ -165,9 +165,13 @@ test_hybrid() {
>>>
>>>      if [ "$pmus" -ne "$cycles_events" ]
>>>      then
>>> -    echo "hybrid test [Found $pmus PMUs but $cycles_events cycles events. Failed]"
>>> -    err=1
>>> -    return
>>> +     # If virtualized the software task-clock event will be used.
>>> +     if ! perf stat -- true 2>&1 | grep -q "task-clock"
>>> +     then
>>> +       echo "hybrid test [Found $pmus PMUs but $cycles_events cycles events. Failed]"
>>> +       err=1
>>> +       return
>>> +     fi
>>>      fi
>>>      echo "hybrid test [Success]"
>>>    }
>>
>> Hi Ian,
>>
>> Isn't the distinction between task-clock and cpu-clock whether the event
>> is per-cpu or not?
>>
>> $ perf stat -C 1 -- true 2>&1 | grep cpu-clock
>>                 1.49 msec cpu-clock       #    0.917 CPUs utilized
>>
>> $ perf stat -- true 2>&1 | grep task-clock
>>                 0.30 msec task-clock      #    0.366 CPUs utilized
>>
>> The test uses per-task mode so this change makes it always pass, even
>> when the number of cycles events doesn't match the PMUs.
> 
> So I'm confused by the test, but it has caused a passing test to be
> broken for me when I run virtualized. The test is checking a cycles
> event is opened on each hybrid PMU, but this is conflated with
> checking perf stat's "default" output. The cycles event will fall back
> to task-clock in per-task mode but we also open a task-clock in
> default mode. Should:
> ```
> if [ "$pmus" -ne "$cycles_events" ]
> ```
> be something like this then:
> ```
> # The expectation is that default output will have a cycles events on
> each hybrid
> # PMU, but in situations with no cycles PMU events, like
> virtualized,this can fall
> # back to task-clock and so the end count may be 0. Fail if neither
> condition holds.
> if [ "$pmus" -ne "$cycles_events" ] && [ "$pmus" -ne "0" ]

Yes it might make sense to skip this part if pmus == 0 then. Namhyung 
suggested to always assume 1 PMU to cover platforms that don't publish 
any PMU with the cpus file, but I suppose that breaks in this case.

If you still add the fallback check for task-clock it covers hybrid, 1 
PMU and virtualized and we could even rename the test to 
test_default_stat_2.

> ```
> 
> Thanks,
> Ian