[PATCH v6 00/19] ARM64 PMU Partitioning

[Colton Lewis <coltonlewis@google.com>]

This series creates a new PMU scheme on ARM, a partitioned PMU that
allows reserving a subset of counters for more direct guest access,
significantly reducing overhead. More details, including performance
benchmarks, can be read in the v1 cover letter linked below.

An overview of what this series accomplishes was presented at KVM
Forum 2025. Slides [1] and video [2] are linked below.

IMPORTANT: This iteration does not yet implement the dynamic counter
reservation approach suggested by Will Deacon in January [3]. I am
working on it, but wanted to send this version first to keep momentum
going and ensure I've addressed all issues besides that.

v6:
* Rebase onto v6.19-rc7

* Drop the reorganization patches I had previously included from Sean
  and Anish and rework without them.

* Inline FGT programming for easier readability

* Change register access path to drop simultaneous writing of the
  virtual and physical registers and write only where the canonical
  state should reside. The PMU register fast path behaves like a
  simple accessor now, relying on generic helpers when needed.

* Related to the previous, drop several patches modifying sys_regs.c
  and incorporate PMOVS and PMEVTYPER into the fast path instead.

* Move the register fast path call to kvm_hyp_handle_sysreg_vhe since
  this feature depends on VHE mode

* Remove the heavyweight access checks from the fast path that had the
  potential to inject an undefined exception. For what checks are
  necessary, just return false and let the normal path handle
  injecting exceptions

* Remove the legacy support for writeable PMCR.N. VMMs must use the
  vCPU attribute to change the number of counters.

* Simplify kvm_pmu_hpmn by relying on kvm_vcpu_on_unsupported_cpu and
  moving HPMN validation of nr_pmu_counters to the ioctl boundary when
  it is set.

* Disable preemption during context swap

* Simplify iteration of counters to context swap by iterating a bitmask

* Clear PMOVS flags during load to avoid the possibility of generating
  a spurious interrupt when writing PMINTEN or PMCNTEN

* Make kvm_pmu_apply_event_filter() hyp safe

* Cleanly separate interrupt handling so the host driver clears the
  overflow flags for the host counters only and KVM handles clearing
  the guest counter flags.

* Ensure the guest PMU state is on hardware before checking hardware
  for the purposes of determining if an overflow should be injected
  into the guest.

* Naming and commit message improvements

* Change uAPI to vCPU device attribute selected when other PMU
  attributes are selected.

* Remove some checks for exceptions when accessing invalid counter
  indices with the Partitioned PMU. Hardware does not guarantee them
  so the Partitioned PMU can't either.

v5:
https://lore.kernel.org/kvmarm/20251209205121.1871534-1-coltonlewis@google.com/

v4:
https://lore.kernel.org/kvmarm/20250714225917.1396543-1-coltonlewis@google.com/

v3:
https://lore.kernel.org/kvm/20250626200459.1153955-1-coltonlewis@google.com/

v2:
https://lore.kernel.org/kvm/20250620221326.1261128-1-coltonlewis@google.com/

v1:
https://lore.kernel.org/kvm/20250602192702.2125115-1-coltonlewis@google.com/

[1] https://gitlab.com/qemu-project/kvm-forum/-/raw/main/_attachments/2025/Optimizing__itvHkhc.pdf
[2] https://www.youtube.com/watch?v=YRzZ8jMIA6M&list=PLW3ep1uCIRfxwmllXTOA2txfDWN6vUOHp&index=9
[3] https://lore.kernel.org/kvmarm/aWjlfl85vSd6sMwT@willie-the-truck/

Colton Lewis (18):
  arm64: cpufeature: Add cpucap for HPMN0
  KVM: arm64: Reorganize PMU functions
  perf: arm_pmuv3: Introduce method to partition the PMU
  perf: arm_pmuv3: Generalize counter bitmasks
  perf: arm_pmuv3: Keep out of guest counter partition
  KVM: arm64: Set up FGT for Partitioned PMU
  KVM: arm64: Define access helpers for PMUSERENR and PMSELR
  KVM: arm64: Write fast path PMU register handlers
  KVM: arm64: Setup MDCR_EL2 to handle a partitioned PMU
  KVM: arm64: Context swap Partitioned PMU guest registers
  KVM: arm64: Enforce PMU event filter at vcpu_load()
  KVM: arm64: Implement lazy PMU context swaps
  perf: arm_pmuv3: Handle IRQs for Partitioned PMU guest counters
  KVM: arm64: Detect overflows for the Partitioned PMU
  KVM: arm64: Add vCPU device attr to partition the PMU
  KVM: selftests: Add find_bit to KVM library
  KVM: arm64: selftests: Add test case for partitioned PMU
  KVM: arm64: selftests: Relax testing for exceptions when partitioned

Marc Zyngier (1):
  KVM: arm64: Reorganize PMU includes

 arch/arm/include/asm/arm_pmuv3.h              |  28 +
 arch/arm64/include/asm/arm_pmuv3.h            |  12 +-
 arch/arm64/include/asm/kvm_host.h             |  17 +-
 arch/arm64/include/asm/kvm_types.h            |   6 +-
 arch/arm64/include/uapi/asm/kvm.h             |   2 +
 arch/arm64/kernel/cpufeature.c                |   8 +
 arch/arm64/kvm/Makefile                       |   2 +-
 arch/arm64/kvm/arm.c                          |   2 +
 arch/arm64/kvm/config.c                       |  41 +-
 arch/arm64/kvm/debug.c                        |  31 +-
 arch/arm64/kvm/hyp/vhe/switch.c               | 240 ++++++
 arch/arm64/kvm/pmu-direct.c                   | 439 +++++++++++
 arch/arm64/kvm/pmu-emul.c                     | 674 +---------------
 arch/arm64/kvm/pmu.c                          | 717 ++++++++++++++++++
 arch/arm64/kvm/sys_regs.c                     |   9 +-
 arch/arm64/tools/cpucaps                      |   1 +
 arch/arm64/tools/sysreg                       |   6 +-
 drivers/perf/arm_pmuv3.c                      | 149 +++-
 include/kvm/arm_pmu.h                         | 126 +++
 include/linux/perf/arm_pmu.h                  |   1 +
 include/linux/perf/arm_pmuv3.h                |  14 +-
 tools/testing/selftests/kvm/Makefile.kvm      |   1 +
 .../selftests/kvm/arm64/vpmu_counter_access.c | 112 ++-
 tools/testing/selftests/kvm/lib/find_bit.c    |   1 +
 24 files changed, 1889 insertions(+), 750 deletions(-)
 create mode 100644 arch/arm64/kvm/pmu-direct.c
 create mode 100644 tools/testing/selftests/kvm/lib/find_bit.c


base-commit: 63804fed149a6750ffd28610c5c1c98cce6bd377
--
2.53.0.rc2.204.g2597b5adb4-goog