Re: KVM: arm64: pmu: Reset sample period on overflow handling

[Aman Priyadarshi <apeureka@amazon.de>]

Hey,

I have been chasing down a regression on guest side related to pmu cycle
counters, and I found that reverting these two patches help:

- 30d97754b2d1 ("KVM: arm/arm64: Re-create event when setting counter
value")
- 8c3252c06516 ("KVM: arm64: pmu: Reset sample period on overflow
handling")

However, I do not yet fully understand the underlying problem.

Regression can be reproduced by running simple userspace processes
consuming 100% of cpus and checking the number of pmu cycles.

```
$ cat foo.c
#include <stdio.h>

int main()
{
    while (1) ;
    return 0;
}


$ gcc -o foo foo.c
$ for x in {0..63}; do ./foo & done
$ sudo perf stat -A -a -e cpu-clock:pppH,cycles -- sleep 10
CPU0        22,397,323,233      cycles                    #    2.240 GHz
CPU1         6,444,741,327      cycles                    #    0.644 GHz
CPU2        17,029,444,425      cycles                    #    1.703 GHz
CPU3         4,298,054,663      cycles                    #    0.430 GHz
CPU4         6,444,769,216      cycles                    #    0.644 GHz
CPU5         6,045,456,891      cycles                    #    0.605 GHz
CPU6         4,298,204,814      cycles                    #    0.430 GHz
CPU7         6,444,346,686      cycles                    #    0.644 GHz
CPU8         4,298,012,726      cycles                    #    0.430 GHz
CPU9         6,445,547,392      cycles                    #    0.645 GHz
CPU10        4,297,996,144      cycles                    #    0.430 GHz
...
```

Expected behavior is to have all cores around 2.5GHz on all CPUs given that
all CPUs are 100% in the guest.

The nonsensical values reported by the pmu counters is only observed on
certain combination of host and guest kernel.

I was able to reproduce it on v5.4.34 and v5.13.0-rc5 running on the host
side, and 4.14.10 on the guest side.

I am running Ubuntu 18.04 cloud image with 4.14.10 kernel headers
installed:

https://cloud-images.ubuntu.com/releases/bionic/release/ubuntu-18.04-server-cloudimg-arm64.img

Note, cloud image comes with 4.15.y kernel installed, which does not show
any regression.

I can also confirm that reverting the mentioned commits on either tree
(v5.4.34 or v5.13.0-rc5) fixes the problem.
```
CPU51          25020026126      cycles                    #    2.497 GHz
CPU52          25020153183      cycles                    #    2.497 GHz
CPU53          25011934659      cycles                    #    2.496 GHz
CPU54          25020521670      cycles                    #    2.497 GHz
CPU55          25015920340      cycles                    #    2.497 GHz
CPU56          25020176871      cycles                    #    2.497 GHz
CPU57          25013646763      cycles                    #    2.497 GHz
CPU58          25020736379      cycles                    #    2.497 GHz
CPU59          25019495291      cycles                    #    2.497 GHz
CPU60          25020988560      cycles                    #    2.497 GHz
CPU61          25021343608      cycles                    #    2.497 GHz
CPU62          25018942029      cycles                    #    2.497 GHz
CPU63          25019514949      cycles                    #    2.497 GHz

      10.020502287 seconds time elapsed

root@ubuntu:~# cat /proc/version  # Guest
Linux version 4.14.10-041410-generic (kernel@kathleen) (gcc version 7.2.0
(Ubuntu/Linaro 7.2.0-6ubuntu1)) #201712291810 SMP Fri Dec 29 18:37:30 UTC
2017
root@ubuntu:~# QEMU: Terminated
ubuntu@ip-10-0-98-166:~/images$ cat /proc/version  # Host
Linux version 5.13.0-rc5 (ubuntu@ip-10-0-98-166) (gcc (Ubuntu 9.3.0-
17ubuntu1~20.04) 9.3.0, GNU ld (GNU Binutils for Ubuntu) 2.34) #2 SMP Mon
Jun 14 21:05:25 UTC 2021
```

Regards,
Aman Priyadarshi




Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879


_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm