* [PATCH v10 00/33] ARM64 PMU Partitioning
@ 2026-11-19 18:51 Colton Lewis
2026-11-19 18:51 ` [PATCH v10 01/33] arm64: cpufeature: Add cpucap for HPMN0 Colton Lewis
` (32 more replies)
0 siblings, 33 replies; 67+ messages in thread
From: Colton Lewis @ 2026-11-19 18:51 UTC (permalink / raw)
To: Marc Zyngier, Oliver Upton
Cc: Catalin Marinas, Will Deacon, Mark Rutland, Peter Zijlstra,
Ingo Molnar, Paolo Bonzini, Russell King, Jonathan Corbet,
Fuad Tabba, Joey Gouly, Suzuki K Poulose, Zenghui Yu, Shuah Khan,
James Clark, Alexandru Elisei, Mingwei Zhang, Ganapatrao Kulkarni,
kvmarm, linux-arm-kernel, kvm, linux-perf-users, linux-kselftest,
linux-doc, linux-kernel
This series creates a new PMU scheme on ARM, a partitioned PMU that
allows reserving a subset of counters for more direct guest access,
significantly reducing overhead. More details, including performance
benchmarks, can be read in the v1 cover letter linked below.
PMU partitioning is controlled via the KVM API using
`KVM_ARM_VCPU_PMU_V3_ENABLE_PARTITION` and
`KVM_ARM_VCPU_PMU_V3_SET_NR_COUNTERS` vCPU device attributes. When
partitioning is enabled for a VM, userspace must explicitly configure a
guest event counter count strictly less than the maximum general-purpose
counters implemented by the PMU (leaving at least one general-purpose
counter for the host) prior to calling `KVM_ARM_VCPU_PMU_V3_INIT`. The
number of general-purpose counters implemented by each host PMU is
exposed in sysfs at
`/sys/bus/event_source/devices/<pmu>/caps/num_counters`. A QEMU patch
demonstrating how to use the uAPI is sent separately.
An overview of what this series accomplishes was presented at KVM
Forum 2025. Slides [1] and video [2] are linked below.
v10:
* Rebase on top of v7.4-rc3.
* Add nested virtualization (`FEAT_NV` / `FEAT_NV2`) support covering
both an Emulated L2 PMU (`vHPMN == N_L1`, `TPM == 1`, where L1 keeps
all `N_L1` counters and emulates L2 via L1 `perf` events) and a
Sub-partitioned L2 PMU (`vHPMN < N_L1`, `TPM == 0`, creating a
three-way split across `0..vHPMN-1` for L2, `vHPMN..N_L1-1` for L1,
and `N_L1..N_phys-1` for L0) (Oliver Upton). Because ARM64 provides
only a single physical `MDCR_EL2.HPMN` field, hardware can split the
PMU into two counter ranges at a time rather than three. Key design
decisions and supporting changes:
- Keep physical `MDCR_EL2.HPMN = N_L1` at all times (`vHPMN` boundary
and event filtering): Lowering physical `HPMN` to `vHPMN` while
running L2 was considered and rejected because it would move L1's
hypervisor counters (`vHPMN..N_L1-1`) into the same physical
`>= HPMN` range as L0's host counters (`N_L1..N_phys-1`), where a
single set of physical `MDCR_EL2.{HPME,HPMD,HLP}` bits governs both
and breaks host/guest isolation; it would also fail for
`vHPMN == 0` on hardware without `FEAT_HPMN0`. Instead, when
`vHPMN < N_L1`, L0 keeps physical `HPMN = N_L1` and sets
`MDCR_EL2.{TPM,TPMCR}` in L2, forwarding `vHPMN..N_L1-1` traps to
L1 and handling `0..vHPMN-1` plus `PMCCNTR_EL0` in L0 directly
against physical registers. This preserves L0 isolation and
supports `vHPMN == 0` on all hardware; L2 events still count
directly in hardware without L0 `perf_event` scheduling, at the
cost of trapping L2 PMU accesses to L0 and rewriting per-counter
filter registers on `L1 <-> L2` transitions. In L1
(`is_hyp_ctxt()`), L0 emulates `vHPMN`, `vMDCR_EL2.HPMD`, and
`vMDCR_EL2.HCCD` via `PMEVTYPER<n>_EL0` and `PMCCFILTR_EL0`.
- Dynamically multiplex `vPMCR_EL0.E` and `vMDCR_EL2.HPME`: Because
both `0..vHPMN-1` and `vHPMN..N_L1-1` sit below physical
`HPMN = N_L1`, hardware gates both with physical `PMCR_EL0.E`.
Rather than unconditionally trapping `PMCR_EL0` and `PMCNTEN*` for
all `FEAT_NV` VMs (which would penalize Emulated L2 PMU where
`vHPMN == N_L1`), L0 dynamically enables `MDCR_EL2.TPMCR` and
`HDFG{R,W}TR_EL2.PMCNTEN` only when `vHPMN < N_L1`. L0 keeps
architectural `PMCR_EL0` and `PMCNTENSET_EL0` in memory, sets
physical `PMCR_EL0.E` to `vPMCR_EL0.E || vMDCR_EL2.HPME`, masks
disabled sub-ranges out of physical `PMCNTENSET_EL0`, and gates
overflow delivery per domain. Trapped `PMSWINC_EL0` writes (also
enabled via `HDFGWTR_EL2` when `vHPMN < N_L1`) flush live registers
and reuse `kvm_pmu_software_increment()` to prevent cross-partition
chaining.
- Bias 32-bit counters on mismatched `vPMCR_EL0.LP != vMDCR_EL2.HLP`
(`FEAT_PMUv3p5`): When `LP == HLP`, physical `PMCR_EL0.LP` matches
both domains directly and `PMEVCNTR<n>_EL0` stays untrapped in L1.
When `vHPMN < N_L1` and `LP != HLP`, rather than ignoring the
mismatch (breaking 32-bit overflow interrupts) or falling back to
`perf_event` emulation mid-flight (requiring reservation teardown),
L0 forces physical `PMCR_EL0.LP = 1` (64-bit overflow), dynamically
traps `HDFG{R,W}TR_EL2.PMEVCNTRn_EL0`, and biases counters in the
32-bit sub-range `2^32` events below 64-bit overflow
(`0xffffffff00000000 | low32`) while tracking their upper-32-bit
offsets and re-biasing outside interrupt/NMI context. Because
`HDFG{R,W}TR_EL2.PMEVCNTRn_EL0` is a single trap bit covering all
event counters (`0..30`), L1 counter reads/writes trap across both
sub-ranges while `LP != HLP`, with zero overhead when `LP == HLP`.
- Baseline nested support (both Emulated and Sub-partitioned L2):
separate implemented and accessible counter masks by using
`nr_pmu_counters` directly in `kvm_pmu_implemented_counter_mask()`,
masking trapped guest reads of `PMCNTEN*`, `PMINTEN*`, and `PMOVS*`
with `kvm_pmu_accessible_counter_mask()`, and ignoring
`MDCR_EL2.HPMN` in userspace `KVM_GET_ONE_REG` / `KVM_SET_ONE_REG`
(`set_pmreg()`, `get_pmreg()`, `get_pmcr()`) so saving/restoring a
nested vCPU paused below EL2 preserves all VM-implemented counters
and true `PMCR_EL0.N`; preserve host counter reservations across
`IN_NESTED_ERET` / `IN_NESTED_EXCEPTION` transitions to avoid host
`perf` reschedules on emulated L2 traps; read live hardware
`PMSELR_EL0` in `check_mdcr_hpmn()` when untrapped; and forward
fine-grained traps for selector-indexed `PMXEVTYPER_EL0` and
`PMXEVCNTR_EL0` to L1.
* Expose the number of general-purpose counters implemented by the PMU
in sysfs under `/sys/bus/event_source/devices/<pmu>/caps/num_counters`
so userspace VMMs can query host PMU counter capacity when configuring
a partitioned PMU (James Clark).
* Make `__vcpu_rmw_sys_reg()` atomic on the local CPU using
`try_cmpxchg_local()` to prevent read-modify-write races with GICv3
pseudo-NMI PMU interrupt handlers (`kvm_pmu_handle_guest_irq()`),
which are not masked by `local_irq_save()` (Sashiko AI Review).
* Construct `mdcr_el2` in a local variable before assigning
`vcpu->arch.mdcr_el2` once in `kvm_arm_setup_mdcr_el2()`, and mask
GICv3 pseudo-NMIs (`local_daif_save()`) across hardware PMU context
switches and `MDCR_EL2` updates (Sashiko AI Review).
* Prioritize kernel-owned pinned event groups (`is_kernel_event()`)
ahead of userspace pinned events in `perf_event_groups_cmp()` /
`perf_less_group_idx()` so kernel-owned per-CPU pinned events (such as
the hardlockup NMI watchdog if toggled at runtime) are scheduled before
userspace pinned events when `perf_pmu_resched_update()` reschedules
events out of guest-reserved counters (James Clark, Sashiko AI Review).
* Move `kvm_vcpu_reload_pmu()` from `arch/arm64/kvm/pmu-emul.c` to
`arch/arm64/kvm/pmu.c` in `KVM: arm64: Reorganize PMU functions`.
* Add `kvm_pmu_is_partitioned()`, `kvm_pmu_guest_counter_mask()`,
`kvm_pmu_handle_guest_irq()`, `kvm_pmu_host_start()`, and
`kvm_pmu_host_stop()` stubs to `arch/arm/include/asm/arm_pmuv3.h` to
fix 32-bit `ARCH=arm` builds (kernel test robot).
* Allow identical VM-wide PMU device attributes
(`KVM_ARM_VCPU_PMU_V3_SET_PMU`, `KVM_ARM_VCPU_PMU_V3_SET_NR_COUNTERS`,
and `KVM_ARM_VCPU_PMU_V3_ENABLE_PARTITION`) to be applied across all
vCPUs when userspace initializes vCPUs sequentially (`CPU_FOREACH`),
validate `kvm_arm_pmu_part_hpmn(n) < max_counters` (requiring
`max_counters >= 2` for `n == 0` without `FEAT_HPMN0`), and restrict
`kvm->arch.supported_cpus` to `arm_pmu->supported_cpus` when
partitioning is enabled.
* Centralize partitioned PMU hardware register dispatch under
`kvm_pmu_direct_reg_read()` / `kvm_pmu_direct_reg_write()` with
preemption disabled (`kvm_pmu_part_on_cpu()`), guard lazy ownership
transitions in `kvm_pmu_part_overflow_status()` with
`scoped_guard(preempt)`, avoid matching `reset_unknown()` poison bits on
`PMCNTENSET_EL0` / `PMINTENSET_EL1` during `KVM_REQ_RELOAD_PMU`, and
schedule `KVM_REQ_RELOAD_PMU` on `KVM_SET_ONE_REG` writes so
`vcpu_put()` does not overwrite userspace register writes (Sashiko AI
Review).
* Support configuring 0 guest event counters (cycle counter only) on
Partitioned PMU hosts without `FEAT_HPMN0` whenever `max_counters >= 2`.
Writing `MDCR_EL2.HPMN = 0` without `FEAT_HPMN0` is architecturally
`CONSTRAINED UNPREDICTABLE`, which would otherwise prevent userspace
from creating a cycle-counter-only partitioned guest on CPUs lacking
`FEAT_HPMN0`. Instead, KVM programs physical `MDCR_EL2.HPMN = 1` via
`kvm_arm_pmu_part_hpmn()`, reserves and quiesces physical counter 0
alongside `PMCCNTR_EL0` so host perf continues using counters `1..N-1`,
keeps `MDCR_EL2.TPMCR = 1` so trapped `PMCR_EL0` reads report `N == 0`,
and enables fine-grained traps on `PMEVCNTRn_EL0`, `PMCNTEN`,
`PMINTEN`, and `PMSWINC_EL0` so counter 0 stays completely hidden
(`UNDEFINED` / RAZ / WI) while `PMCCNTR_EL0` remains untrapped on
`FEAT_FGT` hosts. Also clamp `MDCR_EL2.HPMN` against `nr_pmu_counters`
in `kvm_vcpu_read_pmcr()` and `kvm_pmu_direct_pmcr_read()`, and extend
`vpmu_counter_access` to verify `PMCCFILTR_EL0`, live `PMCCNTR_EL0`
counting, and trapped `UNDEFINED` / RAZ / WI behavior on counter 0 when
`n == 0`.
Tool-assisted development note (generated-content.rst):
Several v10 additions (nested virtualization sub-partitioning,
cycle-counter-only support without `FEAT_HPMN0`, the userspace `HPMN`
masking fix, and cover letter text) were drafted and verified with AI
coding assistance (Gemini) and pre-reviewed with Sashiko AI Review. All
patches were reviewed, built (`arm64`, `CONFIG_ARM_PMU=n`, 32-bit `arm`),
and tested on bare-metal Neoverse V2 and AmpereOne hosts (and under QEMU
TCG for the userspace `HPMN` fix).
v9:
https://lore.kernel.org/kvmarm/20260924172928.2110956-1-coltonlewis@google.com/
v8:
https://lore.kernel.org/kvmarm/20260612192909.1153907-1-coltonlewis@google.com/
v7:
https://lore.kernel.org/kvmarm/20260504211813.1804997-1-coltonlewis@google.com/
v6:
https://lore.kernel.org/kvmarm/20260209221414.2169465-1-coltonlewis@google.com/
v5:
https://lore.kernel.org/kvmarm/20251209205121.1871534-1-coltonlewis@google.com/
v4:
https://lore.kernel.org/kvmarm/20250714225917.1396543-1-coltonlewis@google.com/
v3:
https://lore.kernel.org/kvm/20250626200459.1153955-1-coltonlewis@google.com/
v2:
https://lore.kernel.org/kvm/20250620221326.1261128-1-coltonlewis@google.com/
v1:
https://lore.kernel.org/kvm/20250602192702.2125115-1-coltonlewis@google.com/
[1] https://gitlab.com/qemu-project/kvm-forum/-/raw/main/_attachments/2025/Optimizing__itvHkhc.pdf
[2] https://www.youtube.com/watch?v=YRzZ8jMIA6M&list=PLW3ep1uCIRfxwmllXTOA2txfDWN6vUOHp&index=9
Colton Lewis (32):
arm64: cpufeature: Add cpucap for HPMN0
KVM: arm64: Reorganize PMU functions
KVM: arm64: Use nr_pmu_counters in kvm_pmu_implemented_counter_mask()
KVM: arm64: Ignore HPMN for userspace PMU register access
perf: arm_pmuv3: Generalize counter bitmasks
perf: arm_pmuv3: Move counter allocation mask to per-CPU struct
pmu_hw_events
perf: arm_pmuv3: Check cntr_mask before using pmccntr
perf: arm_pmuv3: Allocate counter indices from high to low
perf: arm_pmuv3: Expose number of general-purpose counters in sysfs
KVM: arm64: Add initial scaffolding for Partitioned PMU
KVM: arm64: Set up FGT for Partitioned PMU
KVM: arm64: Make __vcpu_rmw_sys_reg() atomic
KVM: arm64: Add Partitioned PMU register trap handlers
KVM: arm64: Construct mdcr_el2 in a local variable in
kvm_arm_setup_mdcr_el2()
KVM: arm64: Set up MDCR_EL2 to handle a Partitioned PMU
KVM: arm64: Context swap Partitioned PMU guest registers
KVM: arm64: Enforce PMU event filter at vcpu_load()
perf: Add perf_pmu_resched_update()
perf/core: Prioritize kernel-owned pinned events when scheduling
groups
KVM: arm64: Allow kvm_vcpu_pmu_resync_el0() to resync filters in
process context
KVM: arm64: Apply dynamic guest counter reservations
KVM: arm64: Implement lazy PMU context swaps
perf: arm_pmuv3: Handle IRQs for Partitioned PMU guest counters
KVM: arm64: Detect overflows for the Partitioned PMU
KVM: arm64: Multiplex PMCR_EL0.E and MDCR_EL2.HPME for sub-partitioned
L1
KVM: arm64: Emulate mismatched LP and HLP overflow widths for
sub-partitioned L1
KVM: arm64: Emulate trapped PMSWINC_EL0 writes for Partitioned PMU
KVM: arm64: Forward FGT traps for PMXEVTYPER_EL0 and PMXEVCNTR_EL0
KVM: arm64: Support zero event counters on Partitioned PMU without
HPMN0
KVM: arm64: Add vCPU device attr to partition the PMU
KVM: selftests: Add find_bit to KVM library
KVM: arm64: selftests: Add test case for Partitioned PMU
Marc Zyngier (1):
KVM: arm64: Reorganize PMU includes
Documentation/virt/kvm/devices/vcpu.rst | 57 +-
arch/arm/include/asm/arm_pmuv3.h | 12 +
arch/arm64/include/asm/arm_pmuv3.h | 7 +-
arch/arm64/include/asm/kvm_host.h | 32 +-
arch/arm64/include/asm/kvm_nested.h | 2 +-
arch/arm64/include/asm/kvm_types.h | 6 +-
arch/arm64/include/uapi/asm/kvm.h | 2 +
arch/arm64/kernel/cpufeature.c | 11 +-
arch/arm64/kvm/Makefile | 2 +-
arch/arm64/kvm/arm.c | 4 +-
arch/arm64/kvm/config.c | 76 +-
arch/arm64/kvm/debug.c | 76 +-
arch/arm64/kvm/emulate-nested.c | 25 +-
arch/arm64/kvm/nested.c | 39 +-
arch/arm64/kvm/pmu-direct.c | 1230 +++++++++++++++++
arch/arm64/kvm/pmu-emul.c | 846 ++----------
arch/arm64/kvm/pmu.c | 831 ++++++++++-
arch/arm64/kvm/sys_regs.c | 199 ++-
arch/arm64/tools/cpucaps | 1 +
arch/arm64/tools/sysreg | 6 +-
drivers/perf/arm_pmu.c | 18 +-
drivers/perf/arm_pmuv3.c | 121 +-
include/kvm/arm_pmu.h | 117 ++
include/linux/perf/arm_pmu.h | 2 +
include/linux/perf/arm_pmuv3.h | 14 +-
include/linux/perf_event.h | 3 +
kernel/events/core.c | 61 +-
tools/include/perf/arm_pmuv3.h | 12 +-
tools/testing/selftests/kvm/Makefile.kvm | 1 +
.../selftests/kvm/arm64/vpmu_counter_access.c | 217 ++-
tools/testing/selftests/kvm/lib/find_bit.c | 2 +
31 files changed, 3032 insertions(+), 1000 deletions(-)
create mode 100644 arch/arm64/kvm/pmu-direct.c
create mode 100644 tools/testing/selftests/kvm/lib/find_bit.c
base-commit: aebb7274476403040d582c61478df8f4a7bf9cd3
--
2.56.1.708.g77570a6a1d-goog
^ permalink raw reply [flat|nested] 67+ messages in thread
* [PATCH v10 01/33] arm64: cpufeature: Add cpucap for HPMN0
2026-11-19 18:51 [PATCH v10 00/33] ARM64 PMU Partitioning Colton Lewis
@ 2026-11-19 18:51 ` Colton Lewis
2026-11-19 19:08 ` sashiko-bot
2026-11-19 18:51 ` [PATCH v10 02/33] KVM: arm64: Reorganize PMU includes Colton Lewis
` (31 subsequent siblings)
32 siblings, 1 reply; 67+ messages in thread
From: Colton Lewis @ 2026-11-19 18:51 UTC (permalink / raw)
To: Marc Zyngier, Oliver Upton
Cc: Catalin Marinas, Will Deacon, Mark Rutland, Peter Zijlstra,
Ingo Molnar, Paolo Bonzini, Russell King, Jonathan Corbet,
Fuad Tabba, Joey Gouly, Suzuki K Poulose, Zenghui Yu, Shuah Khan,
James Clark, Alexandru Elisei, Mingwei Zhang, Ganapatrao Kulkarni,
kvmarm, linux-arm-kernel, kvm, linux-perf-users, linux-kselftest,
linux-doc, linux-kernel
Add a capability for FEAT_HPMN0, whether MDCR_EL2.HPMN can specify 0
counters reserved for the guest.
This required changing HPMN0 to an UnsignedEnum in tools/sysreg
because otherwise not all the appropriate macros are generated to add
it to arm64_cpu_capabilities_arm64_features.
Acked-by: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Tested-by: James Clark <james.clark@linaro.org>
Signed-off-by: Colton Lewis <coltonlewis@google.com>
---
arch/arm64/kernel/cpufeature.c | 11 ++++++++++-
arch/arm64/kvm/sys_regs.c | 5 ++++-
arch/arm64/tools/cpucaps | 1 +
arch/arm64/tools/sysreg | 6 +++---
4 files changed, 18 insertions(+), 5 deletions(-)
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index c9c9a6889455c..023b98e6be2d1 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -77,7 +77,7 @@
#include <linux/percpu.h>
#include <linux/sched/isolation.h>
-#include <asm/arm_pmuv3.h>
+#include <linux/perf/arm_pmuv3.h>
#include <asm/cpu.h>
#include <asm/cpufeature.h>
#include <asm/cpu_ops.h>
@@ -566,6 +566,7 @@ static const struct arm64_ftr_bits ftr_id_mmfr0[] = {
};
static const struct arm64_ftr_bits ftr_id_aa64dfr0[] = {
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64DFR0_EL1_HPMN0_SHIFT, 4, 0),
S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64DFR0_EL1_DoubleLock_SHIFT, 4, 0),
ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64DFR0_EL1_PMSVer_SHIFT, 4, 0),
ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64DFR0_EL1_CTX_CMPs_SHIFT, 4, 0),
@@ -730,6 +731,7 @@ static const struct arm64_ftr_bits ftr_id_dfr0[] = {
};
static const struct arm64_ftr_bits ftr_id_dfr1[] = {
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_DFR1_EL1_HPMN0_SHIFT, 4, 0),
S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR1_EL1_MTPMU_SHIFT, 4, 0),
ARM64_FTR_END,
};
@@ -2863,6 +2865,13 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
.matches = has_cpuid_feature,
ARM64_CPUID_FIELDS(ID_AA64MMFR0_EL1, FGT, FGT2)
},
+ {
+ .desc = "HPMN0",
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
+ .capability = ARM64_HAS_HPMN0,
+ .matches = has_cpuid_feature,
+ ARM64_CPUID_FIELDS(ID_AA64DFR0_EL1, HPMN0, IMP)
+ },
#ifdef CONFIG_ARM64_SME
{
.desc = "Scalable Matrix Extension",
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index c7d91138a6d49..04947bfbd0ee5 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -2218,6 +2218,8 @@ static u64 sanitise_id_aa64dfr0_el1(const struct kvm_vcpu *vcpu, u64 val)
if (kvm_vcpu_has_pmu(vcpu))
val |= SYS_FIELD_PREP(ID_AA64DFR0_EL1, PMUVer,
kvm_arm_pmu_get_pmuver_limit());
+ else
+ val &= ~ID_AA64DFR0_EL1_HPMN0_MASK;
/* Hide SPE from guests */
val &= ~ID_AA64DFR0_EL1_PMSVer_MASK;
@@ -3417,7 +3419,8 @@ static const struct sys_reg_desc sys_reg_descs[] = {
ID_AA64DFR0_EL1_DoubleLock_MASK |
ID_AA64DFR0_EL1_WRPs_MASK |
ID_AA64DFR0_EL1_PMUVer_MASK |
- ID_AA64DFR0_EL1_DebugVer_MASK),
+ ID_AA64DFR0_EL1_DebugVer_MASK |
+ ID_AA64DFR0_EL1_HPMN0_MASK),
ID_SANITISED(ID_AA64DFR1_EL1),
ID_UNALLOCATED(5,2),
ID_UNALLOCATED(5,3),
diff --git a/arch/arm64/tools/cpucaps b/arch/arm64/tools/cpucaps
index 3125695122a0a..1603a0dd19764 100644
--- a/arch/arm64/tools/cpucaps
+++ b/arch/arm64/tools/cpucaps
@@ -43,6 +43,7 @@ HAS_GIC_PRIO_MASKING
HAS_GIC_PRIO_RELAXED_SYNC
HAS_ICH_HCR_EL2_TDIR
HAS_HCR_NV1
+HAS_HPMN0
HAS_HCX
HAS_LDAPR
HAS_LPA2
diff --git a/arch/arm64/tools/sysreg b/arch/arm64/tools/sysreg
index 997c62ab72e68..7230a9cd46e53 100644
--- a/arch/arm64/tools/sysreg
+++ b/arch/arm64/tools/sysreg
@@ -1726,9 +1726,9 @@ EndEnum
EndSysreg
Sysreg ID_AA64DFR0_EL1 3 0 0 5 0
-Enum 63:60 HPMN0
- 0b0000 UNPREDICTABLE
- 0b0001 DEF
+UnsignedEnum 63:60 HPMN0
+ 0b0000 NI
+ 0b0001 IMP
EndEnum
UnsignedEnum 59:56 ExtTrcBuff
0b0000 NI
--
2.56.1.708.g77570a6a1d-goog
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v10 02/33] KVM: arm64: Reorganize PMU includes
2026-11-19 18:51 [PATCH v10 00/33] ARM64 PMU Partitioning Colton Lewis
2026-11-19 18:51 ` [PATCH v10 01/33] arm64: cpufeature: Add cpucap for HPMN0 Colton Lewis
@ 2026-11-19 18:51 ` Colton Lewis
2026-11-19 19:00 ` sashiko-bot
2026-11-19 18:51 ` [PATCH v10 03/33] KVM: arm64: Reorganize PMU functions Colton Lewis
` (30 subsequent siblings)
32 siblings, 1 reply; 67+ messages in thread
From: Colton Lewis @ 2026-11-19 18:51 UTC (permalink / raw)
To: Marc Zyngier, Oliver Upton
Cc: Catalin Marinas, Will Deacon, Mark Rutland, Peter Zijlstra,
Ingo Molnar, Paolo Bonzini, Russell King, Jonathan Corbet,
Fuad Tabba, Joey Gouly, Suzuki K Poulose, Zenghui Yu, Shuah Khan,
James Clark, Alexandru Elisei, Mingwei Zhang, Ganapatrao Kulkarni,
kvmarm, linux-arm-kernel, kvm, linux-perf-users, linux-kselftest,
linux-doc, linux-kernel
From: Marc Zyngier <maz@kernel.org>
Including *all* of asm/kvm_host.h in asm/arm_pmuv3.h is a bad idea
because that is much more than arm_pmuv3.h logically needs and creates
a circular dependency that makes it easy to introduce compiler errors
when editing this code.
asm/kvm_host.h includes kvm/arm_pmu.h includes perf/arm_pmuv3.h
includes asm/arm_pmuv3.h includes asm/kvm_host.h
Reorganize the PMU includes to be more sane. In particular:
- Remove the circular dependency by removing the kvm_host.h include
from asm/arm_pmuv3.h since 99% of it isn't needed.
- Move the remaining tiny bit of KVM/PMU interface from kvm_host.h
into arm_pmu.h
- Conditionally on ARM64, include the more targeted arm_pmu.h directly
in the arm_pmuv3.c driver.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Tested-by: James Clark <james.clark@linaro.org>
Signed-off-by: Colton Lewis <coltonlewis@google.com>
---
arch/arm64/include/asm/arm_pmuv3.h | 2 --
arch/arm64/include/asm/kvm_host.h | 14 --------------
drivers/perf/arm_pmuv3.c | 5 +++++
include/kvm/arm_pmu.h | 19 +++++++++++++++++++
4 files changed, 24 insertions(+), 16 deletions(-)
diff --git a/arch/arm64/include/asm/arm_pmuv3.h b/arch/arm64/include/asm/arm_pmuv3.h
index 8a777dec8d88a..cf2b2212e00a2 100644
--- a/arch/arm64/include/asm/arm_pmuv3.h
+++ b/arch/arm64/include/asm/arm_pmuv3.h
@@ -6,8 +6,6 @@
#ifndef __ASM_PMUV3_H
#define __ASM_PMUV3_H
-#include <asm/kvm_host.h>
-
#include <asm/cpufeature.h>
#include <asm/sysreg.h>
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 97332ffcb1300..4683281fcc5d4 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -1534,25 +1534,11 @@ void kvm_arch_vcpu_ctxflush_fp(struct kvm_vcpu *vcpu);
void kvm_arch_vcpu_ctxsync_fp(struct kvm_vcpu *vcpu);
void kvm_arch_vcpu_put_fp(struct kvm_vcpu *vcpu);
-static inline bool kvm_pmu_counter_deferred(struct perf_event_attr *attr)
-{
- return (!has_vhe() && attr->exclude_host);
-}
-
#ifdef CONFIG_KVM
-void kvm_set_pmu_events(u64 set, struct perf_event_attr *attr);
-void kvm_clr_pmu_events(u64 clr);
-bool kvm_set_pmuserenr(u64 val);
void kvm_enable_trbe(void);
void kvm_disable_trbe(void);
void kvm_tracing_set_el1_configuration(u64 trfcr_while_in_guest);
#else
-static inline void kvm_set_pmu_events(u64 set, struct perf_event_attr *attr) {}
-static inline void kvm_clr_pmu_events(u64 clr) {}
-static inline bool kvm_set_pmuserenr(u64 val)
-{
- return false;
-}
static inline void kvm_enable_trbe(void) {}
static inline void kvm_disable_trbe(void) {}
static inline void kvm_tracing_set_el1_configuration(u64 trfcr_while_in_guest) {}
diff --git a/drivers/perf/arm_pmuv3.c b/drivers/perf/arm_pmuv3.c
index 03359e078301f..fbd631ec71a65 100644
--- a/drivers/perf/arm_pmuv3.c
+++ b/drivers/perf/arm_pmuv3.c
@@ -10,6 +10,11 @@
#include <asm/cputype.h>
#include <asm/irq_regs.h>
+
+#if defined(CONFIG_ARM64)
+#include <kvm/arm_pmu.h>
+#endif
+
#include <asm/perf_event.h>
#include <asm/virt.h>
diff --git a/include/kvm/arm_pmu.h b/include/kvm/arm_pmu.h
index 6b4a118d17ca9..93f41ac77c25f 100644
--- a/include/kvm/arm_pmu.h
+++ b/include/kvm/arm_pmu.h
@@ -9,12 +9,22 @@
#include <linux/perf_event.h>
#include <linux/perf/arm_pmuv3.h>
+#include <linux/perf/arm_pmu.h>
#define KVM_ARMV8_PMU_MAX_COUNTERS 32
/* PPI #23 - architecturally specified for GICv5 */
#define KVM_ARMV8_PMU_GICV5_IRQ 0x20000017
+#define kvm_pmu_counter_deferred(attr) \
+ ({ \
+ !has_vhe() && (attr)->exclude_host; \
+ })
+
+struct kvm;
+struct kvm_device_attr;
+struct kvm_vcpu;
+
#if IS_ENABLED(CONFIG_HW_PERF_EVENTS) && IS_ENABLED(CONFIG_KVM)
struct kvm_pmc {
u8 idx; /* index into the pmu->pmc array */
@@ -68,6 +78,9 @@ int kvm_arm_pmu_v3_has_attr(struct kvm_vcpu *vcpu,
int kvm_arm_pmu_v3_enable(struct kvm_vcpu *vcpu);
struct kvm_pmu_events *kvm_get_pmu_events(void);
+void kvm_set_pmu_events(u64 set, struct perf_event_attr *attr);
+void kvm_clr_pmu_events(u64 clr);
+bool kvm_set_pmuserenr(u64 val);
void kvm_vcpu_pmu_restore_guest(struct kvm_vcpu *vcpu);
void kvm_vcpu_pmu_restore_host(struct kvm_vcpu *vcpu);
void kvm_vcpu_pmu_resync_el0(void);
@@ -165,6 +178,12 @@ static inline u64 kvm_pmu_get_pmceid(struct kvm_vcpu *vcpu, bool pmceid1)
#define kvm_vcpu_has_pmu(vcpu) ({ false; })
#define kvm_vcpu_has_pmuv3_strict(vcpu) ({ false; })
static inline void kvm_pmu_update_vcpu_events(struct kvm_vcpu *vcpu) {}
+static inline void kvm_set_pmu_events(u64 set, struct perf_event_attr *attr) {}
+static inline void kvm_clr_pmu_events(u64 clr) {}
+static inline bool kvm_set_pmuserenr(u64 val)
+{
+ return false;
+}
static inline void kvm_vcpu_pmu_restore_guest(struct kvm_vcpu *vcpu) {}
static inline void kvm_vcpu_pmu_restore_host(struct kvm_vcpu *vcpu) {}
static inline void kvm_vcpu_reload_pmu(struct kvm_vcpu *vcpu) {}
--
2.56.1.708.g77570a6a1d-goog
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v10 03/33] KVM: arm64: Reorganize PMU functions
2026-11-19 18:51 [PATCH v10 00/33] ARM64 PMU Partitioning Colton Lewis
2026-11-19 18:51 ` [PATCH v10 01/33] arm64: cpufeature: Add cpucap for HPMN0 Colton Lewis
2026-11-19 18:51 ` [PATCH v10 02/33] KVM: arm64: Reorganize PMU includes Colton Lewis
@ 2026-11-19 18:51 ` Colton Lewis
2026-11-19 18:57 ` sashiko-bot
2026-11-19 18:51 ` [PATCH v10 04/33] KVM: arm64: Use nr_pmu_counters in kvm_pmu_implemented_counter_mask() Colton Lewis
` (29 subsequent siblings)
32 siblings, 1 reply; 67+ messages in thread
From: Colton Lewis @ 2026-11-19 18:51 UTC (permalink / raw)
To: Marc Zyngier, Oliver Upton
Cc: Catalin Marinas, Will Deacon, Mark Rutland, Peter Zijlstra,
Ingo Molnar, Paolo Bonzini, Russell King, Jonathan Corbet,
Fuad Tabba, Joey Gouly, Suzuki K Poulose, Zenghui Yu, Shuah Khan,
James Clark, Alexandru Elisei, Mingwei Zhang, Ganapatrao Kulkarni,
kvmarm, linux-arm-kernel, kvm, linux-perf-users, linux-kselftest,
linux-doc, linux-kernel
A lot of functions in pmu-emul.c aren't specific to the emulated PMU
implementation. Move them to the more appropriate pmu.c file where
shared PMU functions should live.
Tested-by: James Clark <james.clark@linaro.org>
Signed-off-by: Colton Lewis <coltonlewis@google.com>
---
arch/arm64/kvm/pmu-emul.c | 716 +------------------------------------
arch/arm64/kvm/pmu.c | 719 ++++++++++++++++++++++++++++++++++++++
include/kvm/arm_pmu.h | 14 +
3 files changed, 734 insertions(+), 715 deletions(-)
diff --git a/arch/arm64/kvm/pmu-emul.c b/arch/arm64/kvm/pmu-emul.c
index e2b28dbc0451b..ea5908df42890 100644
--- a/arch/arm64/kvm/pmu-emul.c
+++ b/arch/arm64/kvm/pmu-emul.c
@@ -17,19 +17,10 @@
#define PERF_ATTR_CFG1_COUNTER_64BIT BIT(0)
-static LIST_HEAD(arm_pmus);
-static DEFINE_MUTEX(arm_pmus_lock);
-
static void kvm_pmu_create_perf_event(struct kvm_pmc *pmc);
static void kvm_pmu_release_perf_event(struct kvm_pmc *pmc);
static bool kvm_pmu_counter_is_enabled(struct kvm_pmc *pmc);
-bool kvm_supports_guest_pmuv3(void)
-{
- guard(mutex)(&arm_pmus_lock);
- return !list_empty(&arm_pmus);
-}
-
static struct kvm_vcpu *kvm_pmc_to_vcpu(const struct kvm_pmc *pmc)
{
return container_of(pmc, struct kvm_vcpu, arch.pmu.pmc[pmc->idx]);
@@ -40,49 +31,6 @@ static struct kvm_pmc *kvm_vcpu_idx_to_pmc(struct kvm_vcpu *vcpu, int cnt_idx)
return &vcpu->arch.pmu.pmc[cnt_idx];
}
-static u32 __kvm_pmu_event_mask(unsigned int pmuver)
-{
- switch (pmuver) {
- case ID_AA64DFR0_EL1_PMUVer_IMP:
- return GENMASK(9, 0);
- case ID_AA64DFR0_EL1_PMUVer_V3P1:
- case ID_AA64DFR0_EL1_PMUVer_V3P4:
- case ID_AA64DFR0_EL1_PMUVer_V3P5:
- case ID_AA64DFR0_EL1_PMUVer_V3P7:
- return GENMASK(15, 0);
- default:
- /* Shouldn't be here, just for sanity */
- WARN_ONCE(1, "Unknown PMU version %d\n", pmuver);
- fallthrough;
- case ID_AA64DFR0_EL1_PMUVer_NI:
- return 0;
- }
-}
-
-static u32 kvm_pmu_event_mask(struct kvm *kvm)
-{
- u64 dfr0 = kvm_read_vm_id_reg(kvm, SYS_ID_AA64DFR0_EL1);
- u8 pmuver = SYS_FIELD_GET(ID_AA64DFR0_EL1, PMUVer, dfr0);
-
- return __kvm_pmu_event_mask(pmuver);
-}
-
-u64 kvm_pmu_evtyper_mask(struct kvm *kvm)
-{
- u64 mask = ARMV8_PMU_EXCLUDE_EL1 | ARMV8_PMU_EXCLUDE_EL0 |
- kvm_pmu_event_mask(kvm);
-
- if (kvm_has_feat(kvm, ID_AA64PFR0_EL1, EL2, IMP))
- mask |= ARMV8_PMU_INCLUDE_EL2;
-
- if (kvm_has_feat(kvm, ID_AA64PFR0_EL1, EL3, IMP))
- mask |= ARMV8_PMU_EXCLUDE_NS_EL0 |
- ARMV8_PMU_EXCLUDE_NS_EL1 |
- ARMV8_PMU_EXCLUDE_EL3;
-
- return mask;
-}
-
/**
* kvm_pmc_is_64bit - determine if counter is 64bit
* @pmc: counter context
@@ -275,59 +223,6 @@ void kvm_pmu_vcpu_destroy(struct kvm_vcpu *vcpu)
irq_work_sync(&vcpu->arch.pmu.overflow_work);
}
-static u64 kvm_pmu_hyp_counter_mask(struct kvm_vcpu *vcpu)
-{
- unsigned int hpmn, n;
-
- if (!vcpu_has_nv(vcpu))
- return 0;
-
- hpmn = SYS_FIELD_GET(MDCR_EL2, HPMN, __vcpu_sys_reg(vcpu, MDCR_EL2));
- n = vcpu->kvm->arch.nr_pmu_counters;
-
- /*
- * Programming HPMN to a value greater than PMCR_EL0.N is
- * CONSTRAINED UNPREDICTABLE. Make the implementation choice that an
- * UNKNOWN number of counters (in our case, zero) are reserved for EL2.
- */
- if (hpmn >= n)
- return 0;
-
- /*
- * Programming HPMN=0 is CONSTRAINED UNPREDICTABLE if FEAT_HPMN0 isn't
- * implemented. Since KVM's ability to emulate HPMN=0 does not directly
- * depend on hardware (all PMU registers are trapped), make the
- * implementation choice that all counters are included in the second
- * range reserved for EL2/EL3.
- */
- return GENMASK(n - 1, hpmn);
-}
-
-bool kvm_pmu_counter_is_hyp(struct kvm_vcpu *vcpu, unsigned int idx)
-{
- return kvm_pmu_hyp_counter_mask(vcpu) & BIT(idx);
-}
-
-u64 kvm_pmu_accessible_counter_mask(struct kvm_vcpu *vcpu)
-{
- u64 mask = kvm_pmu_implemented_counter_mask(vcpu);
-
- if (!vcpu_has_nv(vcpu) || vcpu_is_el2(vcpu))
- return mask;
-
- return mask & ~kvm_pmu_hyp_counter_mask(vcpu);
-}
-
-u64 kvm_pmu_implemented_counter_mask(struct kvm_vcpu *vcpu)
-{
- u64 val = FIELD_GET(ARMV8_PMU_PMCR_N, kvm_vcpu_read_pmcr(vcpu));
-
- if (val == 0)
- return BIT(ARMV8_PMU_CYCLE_IDX);
- else
- return GENMASK(val - 1, 0) | BIT(ARMV8_PMU_CYCLE_IDX);
-}
-
static void kvm_pmc_enable_perf_event(struct kvm_pmc *pmc)
{
if (!pmc->perf_event) {
@@ -373,7 +268,7 @@ void kvm_pmu_reprogram_counter_mask(struct kvm_vcpu *vcpu, u64 val)
* counter where the values of the global enable control, PMOVSSET_EL0[n], and
* PMINTENSET_EL1[n] are all 1.
*/
-static bool kvm_pmu_overflow_status(struct kvm_vcpu *vcpu)
+bool kvm_pmu_overflow_status(struct kvm_vcpu *vcpu)
{
u64 reg = __vcpu_sys_reg(vcpu, PMOVSSET_EL0);
@@ -396,16 +291,6 @@ static bool kvm_pmu_overflow_status(struct kvm_vcpu *vcpu)
return reg;
}
-static void kvm_pmu_update_state(struct kvm_vcpu *vcpu)
-{
- struct kvm_pmu *pmu = &vcpu->arch.pmu;
-
- if (unlikely(!irqchip_in_kernel(vcpu->kvm)))
- return;
-
- WARN_ON(kvm_vgic_inject_irq(vcpu->kvm, vcpu, pmu->irq_num,
- kvm_pmu_overflow_status(vcpu), pmu));
-}
bool kvm_pmu_should_notify_user(struct kvm_vcpu *vcpu)
{
@@ -426,43 +311,6 @@ bool kvm_pmu_update_run(struct kvm_vcpu *vcpu)
return update;
}
-/**
- * kvm_pmu_flush_hwstate - flush pmu state to cpu
- * @vcpu: The vcpu pointer
- *
- * Check if the PMU has overflowed while we were running in the host, and inject
- * an interrupt if that was the case.
- */
-void kvm_pmu_flush_hwstate(struct kvm_vcpu *vcpu)
-{
- kvm_pmu_update_state(vcpu);
-}
-
-/**
- * kvm_pmu_sync_hwstate - sync pmu state from cpu
- * @vcpu: The vcpu pointer
- *
- * Check if the PMU has overflowed while we were running in the guest, and
- * inject an interrupt if that was the case.
- */
-void kvm_pmu_sync_hwstate(struct kvm_vcpu *vcpu)
-{
- kvm_pmu_update_state(vcpu);
-}
-
-/*
- * When perf interrupt is an NMI, we cannot safely notify the vcpu corresponding
- * to the event.
- * This is why we need a callback to do it once outside of the NMI context.
- */
-static void kvm_pmu_perf_overflow_notify_vcpu(struct irq_work *work)
-{
- struct kvm_vcpu *vcpu;
-
- vcpu = container_of(work, struct kvm_vcpu, arch.pmu.overflow_work);
- kvm_vcpu_kick(vcpu);
-}
-
/*
* Perform an increment on any of the counters described in @mask,
* generating the overflow if required, and propagate it as a chained
@@ -774,568 +622,6 @@ void kvm_pmu_set_counter_event_type(struct kvm_vcpu *vcpu, u64 data,
kvm_pmu_create_perf_event(pmc);
}
-void kvm_host_pmu_init(struct arm_pmu *pmu)
-{
- struct arm_pmu_entry *entry;
-
- /*
- * Check the sanitised PMU version for the system, as KVM does not
- * support implementations where PMUv3 exists on a subset of CPUs.
- */
- if (!pmuv3_implemented(kvm_arm_pmu_get_pmuver_limit()))
- return;
-
- guard(mutex)(&arm_pmus_lock);
-
- entry = kmalloc_obj(*entry);
- if (!entry)
- return;
-
- entry->arm_pmu = pmu;
- list_add_tail(&entry->entry, &arm_pmus);
-}
-
-static struct arm_pmu *kvm_pmu_probe_armpmu(void)
-{
- struct arm_pmu_entry *entry;
- struct arm_pmu *pmu;
- int cpu;
-
- guard(mutex)(&arm_pmus_lock);
-
- /*
- * It is safe to use a stale cpu to iterate the list of PMUs so long as
- * the same value is used for the entirety of the loop. Given this, and
- * the fact that no percpu data is used for the lookup there is no need
- * to disable preemption.
- *
- * It is still necessary to get a valid cpu, though, to probe for the
- * default PMU instance as userspace is not required to specify a PMU
- * type. In order to uphold the preexisting behavior KVM selects the
- * PMU instance for the core during vcpu init. A dependent use
- * case would be a user with disdain of all things big.LITTLE that
- * affines the VMM to a particular cluster of cores.
- *
- * In any case, userspace should just do the sane thing and use the UAPI
- * to select a PMU type directly. But, be wary of the baggage being
- * carried here.
- */
- cpu = raw_smp_processor_id();
- list_for_each_entry(entry, &arm_pmus, entry) {
- pmu = entry->arm_pmu;
-
- if (cpumask_test_cpu(cpu, &pmu->supported_cpus))
- return pmu;
- }
-
- return NULL;
-}
-
-static u64 __compute_pmceid(struct arm_pmu *pmu, bool pmceid1)
-{
- u32 hi[2], lo[2];
-
- bitmap_to_arr32(lo, pmu->pmceid_bitmap, ARMV8_PMUV3_MAX_COMMON_EVENTS);
- bitmap_to_arr32(hi, pmu->pmceid_ext_bitmap, ARMV8_PMUV3_MAX_COMMON_EVENTS);
-
- return ((u64)hi[pmceid1] << 32) | lo[pmceid1];
-}
-
-static u64 compute_pmceid0(struct kvm_vcpu *vcpu)
-{
- u64 val = __compute_pmceid(vcpu->kvm->arch.arm_pmu, 0);
-
- /* always support SW_INCR */
- val |= BIT(ARMV8_PMUV3_PERFCTR_SW_INCR);
- /* always support CHAIN */
- val |= BIT(ARMV8_PMUV3_PERFCTR_CHAIN);
- return val;
-}
-
-static u64 compute_pmceid1(struct kvm_vcpu *vcpu)
-{
- u64 val = __compute_pmceid(vcpu->kvm->arch.arm_pmu, 1);
-
- /*
- * If KVM_ARM_VCPU_PMU_V3_STRICT is not set, PMMIR_EL1 is
- * unconditionally RAZ, so don't advertise STALL_SLOT* events.
- */
- if (!kvm_vcpu_has_pmuv3_strict(vcpu))
- val &= ~(BIT_ULL(ARMV8_PMUV3_PERFCTR_STALL_SLOT - 32) |
- BIT_ULL(ARMV8_PMUV3_PERFCTR_STALL_SLOT_FRONTEND - 32) |
- BIT_ULL(ARMV8_PMUV3_PERFCTR_STALL_SLOT_BACKEND - 32));
-
- return val;
-}
-
-u64 kvm_pmu_get_pmceid(struct kvm_vcpu *vcpu, bool pmceid1)
-{
- unsigned long *bmap = vcpu->kvm->arch.pmu_filter;
- u64 val, mask = 0;
- int base, i, nr_events;
-
- if (!pmceid1) {
- val = compute_pmceid0(vcpu);
- base = 0;
- } else {
- val = compute_pmceid1(vcpu);
- base = 32;
- }
-
- if (!bmap)
- return val;
-
- nr_events = kvm_pmu_event_mask(vcpu->kvm) + 1;
-
- for (i = 0; i < 32; i += 8) {
- u64 byte;
-
- byte = bitmap_get_value8(bmap, base + i);
- mask |= byte << i;
- if (nr_events >= (0x4000 + base + 32)) {
- byte = bitmap_get_value8(bmap, 0x4000 + base + i);
- mask |= byte << (32 + i);
- }
- }
-
- return val & mask;
-}
-
-void kvm_vcpu_reload_pmu(struct kvm_vcpu *vcpu)
-{
- u64 mask = kvm_pmu_implemented_counter_mask(vcpu);
-
- __vcpu_rmw_sys_reg(vcpu, PMOVSSET_EL0, &=, mask);
- __vcpu_rmw_sys_reg(vcpu, PMINTENSET_EL1, &=, mask);
- __vcpu_rmw_sys_reg(vcpu, PMCNTENSET_EL0, &=, mask);
-
- kvm_pmu_reprogram_counter_mask(vcpu, mask);
-}
-
-int kvm_arm_pmu_v3_enable(struct kvm_vcpu *vcpu)
-{
- if (!vcpu->arch.pmu.created)
- return -EINVAL;
-
- /*
- * A valid interrupt configuration for the PMU is either to have a
- * properly configured interrupt number and using an in-kernel
- * irqchip, or to not have an in-kernel GIC and not set an IRQ.
- */
- if (irqchip_in_kernel(vcpu->kvm)) {
- int irq = vcpu->arch.pmu.irq_num;
- /*
- * If we are using an in-kernel vgic, at this point we know
- * the vgic will be initialized, so we can check the PMU irq
- * number against the dimensions of the vgic and make sure
- * it's valid.
- */
- if (!irq_is_ppi(vcpu->kvm, irq) &&
- !vgic_valid_spi(vcpu->kvm, irq))
- return -EINVAL;
- } else if (kvm_arm_pmu_irq_initialized(vcpu)) {
- return -EINVAL;
- }
-
- return 0;
-}
-
-static int kvm_arm_pmu_v3_init(struct kvm_vcpu *vcpu)
-{
- /* Only possible when using KVM_ARM_VCPU_PMU_V3_STRICT */
- if (!vcpu->kvm->arch.arm_pmu)
- return -ENXIO;
-
- if (irqchip_in_kernel(vcpu->kvm)) {
- int ret;
-
- /*
- * If using the PMU with an in-kernel virtual GIC
- * implementation, we require the GIC to be already
- * initialized when initializing the PMU.
- */
- if (!vgic_initialized(vcpu->kvm))
- return -ENODEV;
-
- if (!kvm_arm_pmu_irq_initialized(vcpu)) {
- if (!vgic_is_v5(vcpu->kvm))
- return -ENXIO;
-
- /* Use the architected irq number for GICv5. */
- vcpu->arch.pmu.irq_num = KVM_ARMV8_PMU_GICV5_IRQ;
- }
-
- ret = kvm_vgic_set_owner(vcpu, vcpu->arch.pmu.irq_num,
- &vcpu->arch.pmu);
- if (ret)
- return ret;
- }
-
- init_irq_work(&vcpu->arch.pmu.overflow_work,
- kvm_pmu_perf_overflow_notify_vcpu);
-
- vcpu->arch.pmu.created = true;
- return 0;
-}
-
-/*
- * For one VM the interrupt type must be same for each vcpu.
- * As a PPI, the interrupt number is the same for all vcpus,
- * while as an SPI it must be a separate number per vcpu.
- */
-static bool pmu_irq_is_valid(struct kvm *kvm, int irq)
-{
- unsigned long i;
- struct kvm_vcpu *vcpu;
-
- /* On GICv5, the PMUIRQ is architecturally mandated to be PPI 23 */
- if (vgic_is_v5(kvm) && irq != KVM_ARMV8_PMU_GICV5_IRQ)
- return false;
-
- kvm_for_each_vcpu(i, vcpu, kvm) {
- if (!kvm_arm_pmu_irq_initialized(vcpu))
- continue;
-
- if (irq_is_ppi(vcpu->kvm, irq)) {
- if (vcpu->arch.pmu.irq_num != irq)
- return false;
- } else {
- if (vcpu->arch.pmu.irq_num == irq)
- return false;
- }
- }
-
- return true;
-}
-
-/**
- * kvm_arm_pmu_get_max_counters - Return the max number of PMU counters.
- * @kvm: The kvm pointer
- */
-u8 kvm_arm_pmu_get_max_counters(struct kvm *kvm)
-{
- struct arm_pmu *arm_pmu = kvm->arch.arm_pmu;
-
- /*
- * Under KVM_ARM_VCPU_PMU_V3_STRICT no PMU exists until userspace sets
- * one, so this can be reached before arm_pmu is set. Report no
- * counters in that case.
- */
- if (!arm_pmu)
- return 0;
-
- /*
- * PMUv3 requires that all event counters are capable of counting any
- * event, though the same may not be true of non-PMUv3 hardware.
- */
- if (cpus_have_final_cap(ARM64_WORKAROUND_PMUV3_IMPDEF_TRAPS))
- return 1;
-
- /*
- * The arm_pmu->cntr_mask considers the fixed counter(s) as well.
- * Ignore those and return only the general-purpose counters.
- */
- return bitmap_weight(arm_pmu->cntr_mask, ARMV8_PMU_MAX_GENERAL_COUNTERS);
-}
-
-static void kvm_arm_set_nr_counters(struct kvm *kvm, unsigned int nr)
-{
- kvm->arch.nr_pmu_counters = nr;
-
- /* Reset MDCR_EL2.HPMN behind the vcpus' back... */
- if (test_bit(KVM_ARM_VCPU_HAS_EL2, kvm->arch.vcpu_features)) {
- struct kvm_vcpu *vcpu;
- unsigned long i;
-
- kvm_for_each_vcpu(i, vcpu, kvm) {
- u64 val = __vcpu_sys_reg(vcpu, MDCR_EL2);
- val &= ~MDCR_EL2_HPMN;
- val |= FIELD_PREP(MDCR_EL2_HPMN, kvm->arch.nr_pmu_counters);
- __vcpu_assign_sys_reg(vcpu, MDCR_EL2, val);
- }
- }
-}
-
-static void kvm_arm_set_pmu(struct kvm *kvm, struct arm_pmu *arm_pmu)
-{
- lockdep_assert_held(&kvm->arch.config_lock);
-
- kvm->arch.arm_pmu = arm_pmu;
- kvm_arm_set_nr_counters(kvm, kvm_arm_pmu_get_max_counters(kvm));
-}
-
-/**
- * kvm_arm_set_default_pmu - No PMU set and KVM_ARM_VCPU_PMU_V3_STRICT not
- * set, get the default one.
- * @kvm: The kvm pointer
- *
- * The observant among you will notice that the supported_cpus
- * mask does not get updated for the default PMU even though it
- * is quite possible the selected instance supports only a
- * subset of cores in the system. This is intentional, and
- * upholds the preexisting behavior on heterogeneous systems
- * where vCPUs can be scheduled on any core but the guest
- * counters could stop working.
- */
-int kvm_arm_set_default_pmu(struct kvm *kvm)
-{
- struct arm_pmu *arm_pmu = kvm_pmu_probe_armpmu();
-
- if (!arm_pmu)
- return -ENODEV;
-
- kvm_arm_set_pmu(kvm, arm_pmu);
- return 0;
-}
-
-static int kvm_arm_pmu_v3_set_pmu(struct kvm_vcpu *vcpu, int pmu_id)
-{
- struct kvm *kvm = vcpu->kvm;
- struct arm_pmu_entry *entry;
- struct arm_pmu *arm_pmu;
- int ret = -ENXIO;
-
- lockdep_assert_held(&kvm->arch.config_lock);
- mutex_lock(&arm_pmus_lock);
-
- list_for_each_entry(entry, &arm_pmus, entry) {
- arm_pmu = entry->arm_pmu;
- if (arm_pmu->pmu.type == pmu_id) {
- if (kvm_vm_has_ran_once(kvm) ||
- (kvm->arch.pmu_filter && kvm->arch.arm_pmu != arm_pmu)) {
- ret = -EBUSY;
- break;
- }
-
- kvm_arm_set_pmu(kvm, arm_pmu);
- cpumask_copy(kvm->arch.supported_cpus, &arm_pmu->supported_cpus);
-
- /*
- * Since a specific PMU is explicitly selected,
- * PMMIR_EL1.SLOTS is deterministic to the guest.
- * If KVM_ARM_VCPU_PMU_V3_STRICT is set, snapshot
- * the value to allow the guest to read it.
- */
- if (kvm_vcpu_has_pmuv3_strict(vcpu))
- kvm->arch.pmmir_slots =
- FIELD_GET(ARMV8_PMU_SLOTS,
- arm_pmu->reg_pmmir);
- ret = 0;
- break;
- }
- }
-
- mutex_unlock(&arm_pmus_lock);
- return ret;
-}
-
-static int kvm_arm_pmu_v3_set_nr_counters(struct kvm_vcpu *vcpu, unsigned int n)
-{
- struct kvm *kvm = vcpu->kvm;
-
- if (!kvm->arch.arm_pmu)
- return -EINVAL;
-
- if (n > kvm_arm_pmu_get_max_counters(kvm))
- return -EINVAL;
-
- kvm_arm_set_nr_counters(kvm, n);
- return 0;
-}
-
-int kvm_arm_pmu_v3_set_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
-{
- struct kvm *kvm = vcpu->kvm;
-
- lockdep_assert_held(&kvm->arch.config_lock);
-
- if (!kvm_vcpu_has_pmu(vcpu))
- return -ENODEV;
-
- if (vcpu->arch.pmu.created)
- return -EBUSY;
-
- switch (attr->attr) {
- case KVM_ARM_VCPU_PMU_V3_IRQ: {
- int __user *uaddr = (int __user *)(long)attr->addr;
- int irq;
-
- if (!irqchip_in_kernel(kvm))
- return -EINVAL;
-
- if (get_user(irq, uaddr))
- return -EFAULT;
-
- /* The PMU overflow interrupt can be a PPI or a valid SPI. */
- if (!(irq_is_ppi(vcpu->kvm, irq) || irq_is_spi(vcpu->kvm, irq)))
- return -EINVAL;
-
- if (!pmu_irq_is_valid(kvm, irq))
- return -EINVAL;
-
- if (kvm_arm_pmu_irq_initialized(vcpu))
- return -EBUSY;
-
- kvm_debug("Set kvm ARM PMU irq: %d\n", irq);
- vcpu->arch.pmu.irq_num = irq;
- return 0;
- }
- case KVM_ARM_VCPU_PMU_V3_FILTER: {
- u8 pmuver = kvm_arm_pmu_get_pmuver_limit();
- struct kvm_pmu_event_filter __user *uaddr;
- struct kvm_pmu_event_filter filter;
- int nr_events;
-
- /*
- * Allow userspace to specify an event filter for the entire
- * event range supported by PMUVer of the hardware, rather
- * than the guest's PMUVer for KVM backward compatibility.
- */
- nr_events = __kvm_pmu_event_mask(pmuver) + 1;
-
- uaddr = (struct kvm_pmu_event_filter __user *)(long)attr->addr;
-
- if (copy_from_user(&filter, uaddr, sizeof(filter)))
- return -EFAULT;
-
- if (((u32)filter.base_event + filter.nevents) > nr_events ||
- (filter.action != KVM_PMU_EVENT_ALLOW &&
- filter.action != KVM_PMU_EVENT_DENY))
- return -EINVAL;
-
- if (kvm_vm_has_ran_once(kvm))
- return -EBUSY;
-
- if (!kvm->arch.arm_pmu)
- return -ENXIO;
-
- if (!kvm->arch.pmu_filter) {
- kvm->arch.pmu_filter = bitmap_alloc(nr_events, GFP_KERNEL_ACCOUNT);
- if (!kvm->arch.pmu_filter)
- return -ENOMEM;
-
- /*
- * The default depends on the first applied filter.
- * If it allows events, the default is to deny.
- * Conversely, if the first filter denies a set of
- * events, the default is to allow.
- */
- if (filter.action == KVM_PMU_EVENT_ALLOW)
- bitmap_zero(kvm->arch.pmu_filter, nr_events);
- else
- bitmap_fill(kvm->arch.pmu_filter, nr_events);
- }
-
- if (filter.action == KVM_PMU_EVENT_ALLOW)
- bitmap_set(kvm->arch.pmu_filter, filter.base_event, filter.nevents);
- else
- bitmap_clear(kvm->arch.pmu_filter, filter.base_event, filter.nevents);
-
- return 0;
- }
- case KVM_ARM_VCPU_PMU_V3_SET_PMU: {
- int __user *uaddr = (int __user *)(long)attr->addr;
- int pmu_id;
-
- if (get_user(pmu_id, uaddr))
- return -EFAULT;
-
- return kvm_arm_pmu_v3_set_pmu(vcpu, pmu_id);
- }
- case KVM_ARM_VCPU_PMU_V3_SET_NR_COUNTERS: {
- unsigned int __user *uaddr = (unsigned int __user *)(long)attr->addr;
- unsigned int n;
-
- if (get_user(n, uaddr))
- return -EFAULT;
-
- return kvm_arm_pmu_v3_set_nr_counters(vcpu, n);
- }
- case KVM_ARM_VCPU_PMU_V3_INIT:
- return kvm_arm_pmu_v3_init(vcpu);
- }
-
- return -ENXIO;
-}
-
-int kvm_arm_pmu_v3_get_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
-{
- switch (attr->attr) {
- case KVM_ARM_VCPU_PMU_V3_IRQ: {
- int __user *uaddr = (int __user *)(long)attr->addr;
- int irq;
-
- if (!irqchip_in_kernel(vcpu->kvm))
- return -EINVAL;
-
- if (!kvm_vcpu_has_pmu(vcpu))
- return -ENODEV;
-
- if (!kvm_arm_pmu_irq_initialized(vcpu))
- return -ENXIO;
-
- irq = vcpu->arch.pmu.irq_num;
- return put_user(irq, uaddr);
- }
- }
-
- return -ENXIO;
-}
-
-int kvm_arm_pmu_v3_has_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
-{
- switch (attr->attr) {
- case KVM_ARM_VCPU_PMU_V3_IRQ:
- case KVM_ARM_VCPU_PMU_V3_INIT:
- case KVM_ARM_VCPU_PMU_V3_FILTER:
- case KVM_ARM_VCPU_PMU_V3_SET_PMU:
- case KVM_ARM_VCPU_PMU_V3_SET_NR_COUNTERS:
- if (kvm_vcpu_has_pmu(vcpu))
- return 0;
- }
-
- return -ENXIO;
-}
-
-u8 kvm_arm_pmu_get_pmuver_limit(void)
-{
- unsigned int pmuver;
-
- pmuver = SYS_FIELD_GET(ID_AA64DFR0_EL1, PMUVer,
- read_sanitised_ftr_reg(SYS_ID_AA64DFR0_EL1));
-
- /*
- * Spoof a barebones PMUv3 implementation if the system supports IMPDEF
- * traps of the PMUv3 sysregs
- */
- if (cpus_have_final_cap(ARM64_WORKAROUND_PMUV3_IMPDEF_TRAPS))
- return ID_AA64DFR0_EL1_PMUVer_IMP;
-
- /*
- * Otherwise, treat IMPLEMENTATION DEFINED functionality as
- * unimplemented
- */
- if (pmuver == ID_AA64DFR0_EL1_PMUVer_IMP_DEF)
- return 0;
-
- return min(pmuver, ID_AA64DFR0_EL1_PMUVer_V3P5);
-}
-
-/**
- * kvm_vcpu_read_pmcr - Read PMCR_EL0 register for the vCPU
- * @vcpu: The vcpu pointer
- */
-u64 kvm_vcpu_read_pmcr(struct kvm_vcpu *vcpu)
-{
- u64 pmcr = __vcpu_sys_reg(vcpu, PMCR_EL0);
- u64 n = vcpu->kvm->arch.nr_pmu_counters;
-
- if (vcpu_has_nv(vcpu) && !vcpu_is_el2(vcpu))
- n = FIELD_GET(MDCR_EL2_HPMN, __vcpu_sys_reg(vcpu, MDCR_EL2));
-
- return u64_replace_bits(pmcr, n, ARMV8_PMU_PMCR_N);
-}
-
void kvm_pmu_nested_transition(struct kvm_vcpu *vcpu)
{
bool reprogrammed = false;
diff --git a/arch/arm64/kvm/pmu.c b/arch/arm64/kvm/pmu.c
index 6b48a3d16d0d5..567de460205a0 100644
--- a/arch/arm64/kvm/pmu.c
+++ b/arch/arm64/kvm/pmu.c
@@ -8,8 +8,21 @@
#include <linux/perf/arm_pmu.h>
#include <linux/perf/arm_pmuv3.h>
+#include <kvm/arm_pmu.h>
+#include <kvm/arm_vgic.h>
+
+#include <asm/kvm_emulate.h>
+
+static LIST_HEAD(arm_pmus);
+static DEFINE_MUTEX(arm_pmus_lock);
static DEFINE_PER_CPU(struct kvm_pmu_events, kvm_pmu_events);
+bool kvm_supports_guest_pmuv3(void)
+{
+ guard(mutex)(&arm_pmus_lock);
+ return !list_empty(&arm_pmus);
+}
+
/*
* Given the perf event attributes and system type, determine
* if we are going to need to switch counters at guest entry/exit.
@@ -209,3 +222,709 @@ void kvm_vcpu_pmu_resync_el0(void)
kvm_make_request(KVM_REQ_RESYNC_PMU_EL0, vcpu);
}
+
+void kvm_host_pmu_init(struct arm_pmu *pmu)
+{
+ struct arm_pmu_entry *entry;
+
+ /*
+ * Check the sanitised PMU version for the system, as KVM does not
+ * support implementations where PMUv3 exists on a subset of CPUs.
+ */
+ if (!pmuv3_implemented(kvm_arm_pmu_get_pmuver_limit()))
+ return;
+
+ guard(mutex)(&arm_pmus_lock);
+
+ entry = kmalloc_obj(*entry);
+ if (!entry)
+ return;
+
+ entry->arm_pmu = pmu;
+ list_add_tail(&entry->entry, &arm_pmus);
+}
+
+static struct arm_pmu *kvm_pmu_probe_armpmu(void)
+{
+ struct arm_pmu_entry *entry;
+ struct arm_pmu *pmu;
+ int cpu;
+
+ guard(mutex)(&arm_pmus_lock);
+
+ /*
+ * It is safe to use a stale cpu to iterate the list of PMUs so long as
+ * the same value is used for the entirety of the loop. Given this, and
+ * the fact that no percpu data is used for the lookup there is no need
+ * to disable preemption.
+ *
+ * It is still necessary to get a valid cpu, though, to probe for the
+ * default PMU instance as userspace is not required to specify a PMU
+ * type. In order to uphold the preexisting behavior KVM selects the
+ * PMU instance for the core during vcpu init. A dependent use
+ * case would be a user with disdain of all things big.LITTLE that
+ * affines the VMM to a particular cluster of cores.
+ *
+ * In any case, userspace should just do the sane thing and use the UAPI
+ * to select a PMU type directly. But, be wary of the baggage being
+ * carried here.
+ */
+ cpu = raw_smp_processor_id();
+ list_for_each_entry(entry, &arm_pmus, entry) {
+ pmu = entry->arm_pmu;
+
+ if (cpumask_test_cpu(cpu, &pmu->supported_cpus))
+ return pmu;
+ }
+
+ return NULL;
+}
+
+static u64 __compute_pmceid(struct arm_pmu *pmu, bool pmceid1)
+{
+ u32 hi[2], lo[2];
+
+ bitmap_to_arr32(lo, pmu->pmceid_bitmap, ARMV8_PMUV3_MAX_COMMON_EVENTS);
+ bitmap_to_arr32(hi, pmu->pmceid_ext_bitmap, ARMV8_PMUV3_MAX_COMMON_EVENTS);
+
+ return ((u64)hi[pmceid1] << 32) | lo[pmceid1];
+}
+
+static u64 compute_pmceid0(struct kvm_vcpu *vcpu)
+{
+ u64 val = __compute_pmceid(vcpu->kvm->arch.arm_pmu, 0);
+
+ /* always support SW_INCR */
+ val |= BIT(ARMV8_PMUV3_PERFCTR_SW_INCR);
+ /* always support CHAIN */
+ val |= BIT(ARMV8_PMUV3_PERFCTR_CHAIN);
+ return val;
+}
+
+static u64 compute_pmceid1(struct kvm_vcpu *vcpu)
+{
+ u64 val = __compute_pmceid(vcpu->kvm->arch.arm_pmu, 1);
+
+ /*
+ * If KVM_ARM_VCPU_PMU_V3_STRICT is not set, PMMIR_EL1 is
+ * unconditionally RAZ, so don't advertise STALL_SLOT* events.
+ */
+ if (!kvm_vcpu_has_pmuv3_strict(vcpu))
+ val &= ~(BIT_ULL(ARMV8_PMUV3_PERFCTR_STALL_SLOT - 32) |
+ BIT_ULL(ARMV8_PMUV3_PERFCTR_STALL_SLOT_FRONTEND - 32) |
+ BIT_ULL(ARMV8_PMUV3_PERFCTR_STALL_SLOT_BACKEND - 32));
+
+ return val;
+}
+
+u64 kvm_pmu_get_pmceid(struct kvm_vcpu *vcpu, bool pmceid1)
+{
+ unsigned long *bmap = vcpu->kvm->arch.pmu_filter;
+ u64 val, mask = 0;
+ int base, i, nr_events;
+
+ if (!pmceid1) {
+ val = compute_pmceid0(vcpu);
+ base = 0;
+ } else {
+ val = compute_pmceid1(vcpu);
+ base = 32;
+ }
+
+ if (!bmap)
+ return val;
+
+ nr_events = kvm_pmu_event_mask(vcpu->kvm) + 1;
+
+ for (i = 0; i < 32; i += 8) {
+ u64 byte;
+
+ byte = bitmap_get_value8(bmap, base + i);
+ mask |= byte << i;
+ if (nr_events >= (0x4000 + base + 32)) {
+ byte = bitmap_get_value8(bmap, 0x4000 + base + i);
+ mask |= byte << (32 + i);
+ }
+ }
+
+ return val & mask;
+}
+
+/*
+ * When perf interrupt is an NMI, we cannot safely notify the vcpu corresponding
+ * to the event.
+ * This is why we need a callback to do it once outside of the NMI context.
+ */
+static void kvm_pmu_perf_overflow_notify_vcpu(struct irq_work *work)
+{
+ struct kvm_vcpu *vcpu;
+
+ vcpu = container_of(work, struct kvm_vcpu, arch.pmu.overflow_work);
+ kvm_vcpu_kick(vcpu);
+}
+
+static u32 __kvm_pmu_event_mask(unsigned int pmuver)
+{
+ switch (pmuver) {
+ case ID_AA64DFR0_EL1_PMUVer_IMP:
+ return GENMASK(9, 0);
+ case ID_AA64DFR0_EL1_PMUVer_V3P1:
+ case ID_AA64DFR0_EL1_PMUVer_V3P4:
+ case ID_AA64DFR0_EL1_PMUVer_V3P5:
+ case ID_AA64DFR0_EL1_PMUVer_V3P7:
+ return GENMASK(15, 0);
+ default:
+ /* Shouldn't be here, just for sanity */
+ WARN_ONCE(1, "Unknown PMU version %d\n", pmuver);
+ fallthrough;
+ case ID_AA64DFR0_EL1_PMUVer_NI:
+ return 0;
+ }
+}
+
+u32 kvm_pmu_event_mask(struct kvm *kvm)
+{
+ u64 dfr0 = kvm_read_vm_id_reg(kvm, SYS_ID_AA64DFR0_EL1);
+ u8 pmuver = SYS_FIELD_GET(ID_AA64DFR0_EL1, PMUVer, dfr0);
+
+ return __kvm_pmu_event_mask(pmuver);
+}
+
+u64 kvm_pmu_evtyper_mask(struct kvm *kvm)
+{
+ u64 mask = ARMV8_PMU_EXCLUDE_EL1 | ARMV8_PMU_EXCLUDE_EL0 |
+ kvm_pmu_event_mask(kvm);
+
+ if (kvm_has_feat(kvm, ID_AA64PFR0_EL1, EL2, IMP))
+ mask |= ARMV8_PMU_INCLUDE_EL2;
+
+ if (kvm_has_feat(kvm, ID_AA64PFR0_EL1, EL3, IMP))
+ mask |= ARMV8_PMU_EXCLUDE_NS_EL0 |
+ ARMV8_PMU_EXCLUDE_NS_EL1 |
+ ARMV8_PMU_EXCLUDE_EL3;
+
+ return mask;
+}
+
+static void kvm_pmu_update_state(struct kvm_vcpu *vcpu)
+{
+ struct kvm_pmu *pmu = &vcpu->arch.pmu;
+
+ if (unlikely(!irqchip_in_kernel(vcpu->kvm)))
+ return;
+
+ WARN_ON(kvm_vgic_inject_irq(vcpu->kvm, vcpu, pmu->irq_num,
+ kvm_pmu_overflow_status(vcpu), pmu));
+}
+
+/**
+ * kvm_pmu_flush_hwstate - flush pmu state to cpu
+ * @vcpu: The vcpu pointer
+ *
+ * Check if the PMU has overflowed while we were running in the host, and inject
+ * an interrupt if that was the case.
+ */
+void kvm_pmu_flush_hwstate(struct kvm_vcpu *vcpu)
+{
+ kvm_pmu_update_state(vcpu);
+}
+
+/**
+ * kvm_pmu_sync_hwstate - sync pmu state from cpu
+ * @vcpu: The vcpu pointer
+ *
+ * Check if the PMU has overflowed while we were running in the guest, and
+ * inject an interrupt if that was the case.
+ */
+void kvm_pmu_sync_hwstate(struct kvm_vcpu *vcpu)
+{
+ kvm_pmu_update_state(vcpu);
+}
+
+int kvm_arm_pmu_v3_enable(struct kvm_vcpu *vcpu)
+{
+ if (!vcpu->arch.pmu.created)
+ return -EINVAL;
+
+ /*
+ * A valid interrupt configuration for the PMU is either to have a
+ * properly configured interrupt number and using an in-kernel
+ * irqchip, or to not have an in-kernel GIC and not set an IRQ.
+ */
+ if (irqchip_in_kernel(vcpu->kvm)) {
+ int irq = vcpu->arch.pmu.irq_num;
+ /*
+ * If we are using an in-kernel vgic, at this point we know
+ * the vgic will be initialized, so we can check the PMU irq
+ * number against the dimensions of the vgic and make sure
+ * it's valid.
+ */
+ if (!irq_is_ppi(vcpu->kvm, irq) && !vgic_valid_spi(vcpu->kvm, irq))
+ return -EINVAL;
+ } else if (kvm_arm_pmu_irq_initialized(vcpu)) {
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int kvm_arm_pmu_v3_init(struct kvm_vcpu *vcpu)
+{
+ /* Only possible when using KVM_ARM_VCPU_PMU_V3_STRICT */
+ if (!vcpu->kvm->arch.arm_pmu)
+ return -ENXIO;
+
+ if (irqchip_in_kernel(vcpu->kvm)) {
+ int ret;
+
+ /*
+ * If using the PMU with an in-kernel virtual GIC
+ * implementation, we require the GIC to be already
+ * initialized when initializing the PMU.
+ */
+ if (!vgic_initialized(vcpu->kvm))
+ return -ENODEV;
+
+ if (!kvm_arm_pmu_irq_initialized(vcpu)) {
+ if (!vgic_is_v5(vcpu->kvm))
+ return -ENXIO;
+
+ /* Use the architected irq number for GICv5. */
+ vcpu->arch.pmu.irq_num = KVM_ARMV8_PMU_GICV5_IRQ;
+ }
+
+ ret = kvm_vgic_set_owner(vcpu, vcpu->arch.pmu.irq_num,
+ &vcpu->arch.pmu);
+ if (ret)
+ return ret;
+ }
+
+ init_irq_work(&vcpu->arch.pmu.overflow_work,
+ kvm_pmu_perf_overflow_notify_vcpu);
+
+ vcpu->arch.pmu.created = true;
+ return 0;
+}
+
+/*
+ * For one VM the interrupt type must be same for each vcpu.
+ * As a PPI, the interrupt number is the same for all vcpus,
+ * while as an SPI it must be a separate number per vcpu.
+ */
+static bool pmu_irq_is_valid(struct kvm *kvm, int irq)
+{
+ unsigned long i;
+ struct kvm_vcpu *vcpu;
+
+ /* On GICv5, the PMUIRQ is architecturally mandated to be PPI 23 */
+ if (vgic_is_v5(kvm) && irq != KVM_ARMV8_PMU_GICV5_IRQ)
+ return false;
+
+ kvm_for_each_vcpu(i, vcpu, kvm) {
+ if (!kvm_arm_pmu_irq_initialized(vcpu))
+ continue;
+
+ if (irq_is_ppi(kvm, irq)) {
+ if (vcpu->arch.pmu.irq_num != irq)
+ return false;
+ } else {
+ if (vcpu->arch.pmu.irq_num == irq)
+ return false;
+ }
+ }
+
+ return true;
+}
+
+/**
+ * kvm_arm_pmu_get_max_counters - Return the max number of PMU counters.
+ * @kvm: The kvm pointer
+ */
+u8 kvm_arm_pmu_get_max_counters(struct kvm *kvm)
+{
+ struct arm_pmu *arm_pmu = kvm->arch.arm_pmu;
+
+ /*
+ * Under KVM_ARM_VCPU_PMU_V3_STRICT no PMU exists until userspace sets
+ * one, so this can be reached before arm_pmu is set. Report no
+ * counters in that case.
+ */
+ if (!arm_pmu)
+ return 0;
+
+ /*
+ * PMUv3 requires that all event counters are capable of counting any
+ * event, though the same may not be true of non-PMUv3 hardware.
+ */
+ if (cpus_have_final_cap(ARM64_WORKAROUND_PMUV3_IMPDEF_TRAPS))
+ return 1;
+
+ /*
+ * The arm_pmu->cntr_mask considers the fixed counter(s) as well.
+ * Ignore those and return only the general-purpose counters.
+ */
+ return bitmap_weight(arm_pmu->cntr_mask, ARMV8_PMU_MAX_GENERAL_COUNTERS);
+}
+
+static void kvm_arm_set_nr_counters(struct kvm *kvm, unsigned int nr)
+{
+ kvm->arch.nr_pmu_counters = nr;
+
+ /* Reset MDCR_EL2.HPMN behind the vcpus' back... */
+ if (test_bit(KVM_ARM_VCPU_HAS_EL2, kvm->arch.vcpu_features)) {
+ struct kvm_vcpu *vcpu;
+ unsigned long i;
+
+ kvm_for_each_vcpu(i, vcpu, kvm) {
+ u64 val = __vcpu_sys_reg(vcpu, MDCR_EL2);
+
+ val &= ~MDCR_EL2_HPMN;
+ val |= FIELD_PREP(MDCR_EL2_HPMN, kvm->arch.nr_pmu_counters);
+ __vcpu_assign_sys_reg(vcpu, MDCR_EL2, val);
+ }
+ }
+}
+
+static void kvm_arm_set_pmu(struct kvm *kvm, struct arm_pmu *arm_pmu)
+{
+ lockdep_assert_held(&kvm->arch.config_lock);
+
+ kvm->arch.arm_pmu = arm_pmu;
+ kvm_arm_set_nr_counters(kvm, kvm_arm_pmu_get_max_counters(kvm));
+}
+
+/**
+ * kvm_arm_set_default_pmu - No PMU set and KVM_ARM_VCPU_PMU_V3_STRICT not
+ * set, get the default one.
+ * @kvm: The kvm pointer
+ *
+ * The observant among you will notice that the supported_cpus
+ * mask does not get updated for the default PMU even though it
+ * is quite possible the selected instance supports only a
+ * subset of cores in the system. This is intentional, and
+ * upholds the preexisting behavior on heterogeneous systems
+ * where vCPUs can be scheduled on any core but the guest
+ * counters could stop working.
+ */
+int kvm_arm_set_default_pmu(struct kvm *kvm)
+{
+ struct arm_pmu *arm_pmu = kvm_pmu_probe_armpmu();
+
+ if (!arm_pmu)
+ return -ENODEV;
+
+ kvm_arm_set_pmu(kvm, arm_pmu);
+ return 0;
+}
+
+static int kvm_arm_pmu_v3_set_pmu(struct kvm_vcpu *vcpu, int pmu_id)
+{
+ struct kvm *kvm = vcpu->kvm;
+ struct arm_pmu_entry *entry;
+ struct arm_pmu *arm_pmu;
+ int ret = -ENXIO;
+
+ lockdep_assert_held(&kvm->arch.config_lock);
+ mutex_lock(&arm_pmus_lock);
+
+ list_for_each_entry(entry, &arm_pmus, entry) {
+ arm_pmu = entry->arm_pmu;
+ if (arm_pmu->pmu.type == pmu_id) {
+ if (kvm_vm_has_ran_once(kvm) ||
+ (kvm->arch.pmu_filter && kvm->arch.arm_pmu != arm_pmu)) {
+ ret = -EBUSY;
+ break;
+ }
+
+ kvm_arm_set_pmu(kvm, arm_pmu);
+ cpumask_copy(kvm->arch.supported_cpus, &arm_pmu->supported_cpus);
+
+ /*
+ * Since a specific PMU is explicitly selected,
+ * PMMIR_EL1.SLOTS is deterministic to the guest.
+ * If KVM_ARM_VCPU_PMU_V3_STRICT is set, snapshot
+ * the value to allow the guest to read it.
+ */
+ if (kvm_vcpu_has_pmuv3_strict(vcpu))
+ kvm->arch.pmmir_slots =
+ FIELD_GET(ARMV8_PMU_SLOTS,
+ arm_pmu->reg_pmmir);
+ ret = 0;
+ break;
+ }
+ }
+
+ mutex_unlock(&arm_pmus_lock);
+ return ret;
+}
+
+static int kvm_arm_pmu_v3_set_nr_counters(struct kvm_vcpu *vcpu, unsigned int n)
+{
+ struct kvm *kvm = vcpu->kvm;
+
+ if (!kvm->arch.arm_pmu)
+ return -EINVAL;
+
+ if (n > kvm_arm_pmu_get_max_counters(kvm))
+ return -EINVAL;
+
+ kvm_arm_set_nr_counters(kvm, n);
+ return 0;
+}
+
+int kvm_arm_pmu_v3_set_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
+{
+ struct kvm *kvm = vcpu->kvm;
+
+ lockdep_assert_held(&kvm->arch.config_lock);
+
+ if (!kvm_vcpu_has_pmu(vcpu))
+ return -ENODEV;
+
+ if (vcpu->arch.pmu.created)
+ return -EBUSY;
+
+ switch (attr->attr) {
+ case KVM_ARM_VCPU_PMU_V3_IRQ: {
+ int __user *uaddr = (int __user *)(long)attr->addr;
+ int irq;
+
+ if (!irqchip_in_kernel(kvm))
+ return -EINVAL;
+
+ if (get_user(irq, uaddr))
+ return -EFAULT;
+
+ /* The PMU overflow interrupt can be a PPI or a valid SPI. */
+ if (!(irq_is_ppi(kvm, irq) || irq_is_spi(kvm, irq)))
+ return -EINVAL;
+
+ if (!pmu_irq_is_valid(kvm, irq))
+ return -EINVAL;
+
+ if (kvm_arm_pmu_irq_initialized(vcpu))
+ return -EBUSY;
+
+ kvm_debug("Set kvm ARM PMU irq: %d\n", irq);
+ vcpu->arch.pmu.irq_num = irq;
+ return 0;
+ }
+ case KVM_ARM_VCPU_PMU_V3_FILTER: {
+ u8 pmuver = kvm_arm_pmu_get_pmuver_limit();
+ struct kvm_pmu_event_filter __user *uaddr;
+ struct kvm_pmu_event_filter filter;
+ int nr_events;
+
+ /*
+ * Allow userspace to specify an event filter for the entire
+ * event range supported by PMUVer of the hardware, rather
+ * than the guest's PMUVer for KVM backward compatibility.
+ */
+ nr_events = __kvm_pmu_event_mask(pmuver) + 1;
+
+ uaddr = (struct kvm_pmu_event_filter __user *)(long)attr->addr;
+
+ if (copy_from_user(&filter, uaddr, sizeof(filter)))
+ return -EFAULT;
+
+ if (((u32)filter.base_event + filter.nevents) > nr_events ||
+ (filter.action != KVM_PMU_EVENT_ALLOW &&
+ filter.action != KVM_PMU_EVENT_DENY))
+ return -EINVAL;
+
+ if (kvm_vm_has_ran_once(kvm))
+ return -EBUSY;
+
+ if (!kvm->arch.arm_pmu)
+ return -ENXIO;
+
+ if (!kvm->arch.pmu_filter) {
+ kvm->arch.pmu_filter = bitmap_alloc(nr_events, GFP_KERNEL_ACCOUNT);
+ if (!kvm->arch.pmu_filter)
+ return -ENOMEM;
+
+ /*
+ * The default depends on the first applied filter.
+ * If it allows events, the default is to deny.
+ * Conversely, if the first filter denies a set of
+ * events, the default is to allow.
+ */
+ if (filter.action == KVM_PMU_EVENT_ALLOW)
+ bitmap_zero(kvm->arch.pmu_filter, nr_events);
+ else
+ bitmap_fill(kvm->arch.pmu_filter, nr_events);
+ }
+
+ if (filter.action == KVM_PMU_EVENT_ALLOW)
+ bitmap_set(kvm->arch.pmu_filter, filter.base_event, filter.nevents);
+ else
+ bitmap_clear(kvm->arch.pmu_filter, filter.base_event, filter.nevents);
+
+ return 0;
+ }
+ case KVM_ARM_VCPU_PMU_V3_SET_PMU: {
+ int __user *uaddr = (int __user *)(long)attr->addr;
+ int pmu_id;
+
+ if (get_user(pmu_id, uaddr))
+ return -EFAULT;
+
+ return kvm_arm_pmu_v3_set_pmu(vcpu, pmu_id);
+ }
+ case KVM_ARM_VCPU_PMU_V3_SET_NR_COUNTERS: {
+ unsigned int __user *uaddr = (unsigned int __user *)(long)attr->addr;
+ unsigned int n;
+
+ if (get_user(n, uaddr))
+ return -EFAULT;
+
+ return kvm_arm_pmu_v3_set_nr_counters(vcpu, n);
+ }
+ case KVM_ARM_VCPU_PMU_V3_INIT:
+ return kvm_arm_pmu_v3_init(vcpu);
+ }
+
+ return -ENXIO;
+}
+
+int kvm_arm_pmu_v3_get_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
+{
+ switch (attr->attr) {
+ case KVM_ARM_VCPU_PMU_V3_IRQ: {
+ int __user *uaddr = (int __user *)(long)attr->addr;
+ int irq;
+
+ if (!irqchip_in_kernel(vcpu->kvm))
+ return -EINVAL;
+
+ if (!kvm_vcpu_has_pmu(vcpu))
+ return -ENODEV;
+
+ if (!kvm_arm_pmu_irq_initialized(vcpu))
+ return -ENXIO;
+
+ irq = vcpu->arch.pmu.irq_num;
+ return put_user(irq, uaddr);
+ }
+ }
+
+ return -ENXIO;
+}
+
+int kvm_arm_pmu_v3_has_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
+{
+ switch (attr->attr) {
+ case KVM_ARM_VCPU_PMU_V3_IRQ:
+ case KVM_ARM_VCPU_PMU_V3_INIT:
+ case KVM_ARM_VCPU_PMU_V3_FILTER:
+ case KVM_ARM_VCPU_PMU_V3_SET_PMU:
+ case KVM_ARM_VCPU_PMU_V3_SET_NR_COUNTERS:
+ if (kvm_vcpu_has_pmu(vcpu))
+ return 0;
+ }
+
+ return -ENXIO;
+}
+
+u8 kvm_arm_pmu_get_pmuver_limit(void)
+{
+ unsigned int pmuver;
+
+ pmuver = SYS_FIELD_GET(ID_AA64DFR0_EL1, PMUVer,
+ read_sanitised_ftr_reg(SYS_ID_AA64DFR0_EL1));
+
+ /*
+ * Spoof a barebones PMUv3 implementation if the system supports IMPDEF
+ * traps of the PMUv3 sysregs
+ */
+ if (cpus_have_final_cap(ARM64_WORKAROUND_PMUV3_IMPDEF_TRAPS))
+ return ID_AA64DFR0_EL1_PMUVer_IMP;
+
+ /*
+ * Otherwise, treat IMPLEMENTATION DEFINED functionality as
+ * unimplemented
+ */
+ if (pmuver == ID_AA64DFR0_EL1_PMUVer_IMP_DEF)
+ return 0;
+
+ return min(pmuver, ID_AA64DFR0_EL1_PMUVer_V3P5);
+}
+
+u64 kvm_pmu_implemented_counter_mask(struct kvm_vcpu *vcpu)
+{
+ u64 val = FIELD_GET(ARMV8_PMU_PMCR_N, kvm_vcpu_read_pmcr(vcpu));
+
+ if (val == 0)
+ return BIT(ARMV8_PMU_CYCLE_IDX);
+ else
+ return GENMASK(val - 1, 0) | BIT(ARMV8_PMU_CYCLE_IDX);
+}
+
+u64 kvm_pmu_hyp_counter_mask(struct kvm_vcpu *vcpu)
+{
+ unsigned int hpmn, n;
+
+ if (!vcpu_has_nv(vcpu))
+ return 0;
+
+ hpmn = SYS_FIELD_GET(MDCR_EL2, HPMN, __vcpu_sys_reg(vcpu, MDCR_EL2));
+ n = vcpu->kvm->arch.nr_pmu_counters;
+
+ /*
+ * Programming HPMN to a value greater than PMCR_EL0.N is
+ * CONSTRAINED UNPREDICTABLE. Make the implementation choice that an
+ * UNKNOWN number of counters (in our case, zero) are reserved for EL2.
+ */
+ if (hpmn >= n)
+ return 0;
+
+ /*
+ * Programming HPMN=0 is CONSTRAINED UNPREDICTABLE if FEAT_HPMN0 isn't
+ * implemented. Since KVM's ability to emulate HPMN=0 does not directly
+ * depend on hardware (all PMU registers are trapped), make the
+ * implementation choice that all counters are included in the second
+ * range reserved for EL2/EL3.
+ */
+ return GENMASK(n - 1, hpmn);
+}
+
+bool kvm_pmu_counter_is_hyp(struct kvm_vcpu *vcpu, unsigned int idx)
+{
+ return kvm_pmu_hyp_counter_mask(vcpu) & BIT(idx);
+}
+
+u64 kvm_pmu_accessible_counter_mask(struct kvm_vcpu *vcpu)
+{
+ u64 mask = kvm_pmu_implemented_counter_mask(vcpu);
+
+ if (!vcpu_has_nv(vcpu) || vcpu_is_el2(vcpu))
+ return mask;
+
+ return mask & ~kvm_pmu_hyp_counter_mask(vcpu);
+}
+
+/**
+ * kvm_vcpu_read_pmcr - Read PMCR_EL0 register for the vCPU
+ * @vcpu: The vcpu pointer
+ */
+u64 kvm_vcpu_read_pmcr(struct kvm_vcpu *vcpu)
+{
+ u64 pmcr = __vcpu_sys_reg(vcpu, PMCR_EL0);
+ u64 n = vcpu->kvm->arch.nr_pmu_counters;
+
+ if (vcpu_has_nv(vcpu) && !vcpu_is_el2(vcpu))
+ n = FIELD_GET(MDCR_EL2_HPMN, __vcpu_sys_reg(vcpu, MDCR_EL2));
+
+ return u64_replace_bits(pmcr, n, ARMV8_PMU_PMCR_N);
+}
+
+void kvm_vcpu_reload_pmu(struct kvm_vcpu *vcpu)
+{
+ u64 mask = kvm_pmu_implemented_counter_mask(vcpu);
+
+ __vcpu_rmw_sys_reg(vcpu, PMOVSSET_EL0, &=, mask);
+ __vcpu_rmw_sys_reg(vcpu, PMINTENSET_EL1, &=, mask);
+ __vcpu_rmw_sys_reg(vcpu, PMCNTENSET_EL0, &=, mask);
+
+ kvm_pmu_reprogram_counter_mask(vcpu, mask);
+}
diff --git a/include/kvm/arm_pmu.h b/include/kvm/arm_pmu.h
index 93f41ac77c25f..8d51cf40182d6 100644
--- a/include/kvm/arm_pmu.h
+++ b/include/kvm/arm_pmu.h
@@ -26,6 +26,7 @@ struct kvm_device_attr;
struct kvm_vcpu;
#if IS_ENABLED(CONFIG_HW_PERF_EVENTS) && IS_ENABLED(CONFIG_KVM)
+
struct kvm_pmc {
u8 idx; /* index into the pmu->pmc array */
struct perf_event *perf_event;
@@ -55,13 +56,16 @@ u64 kvm_pmu_get_counter_value(struct kvm_vcpu *vcpu, u64 select_idx);
void kvm_pmu_set_counter_value(struct kvm_vcpu *vcpu, u64 select_idx, u64 val);
void kvm_pmu_set_counter_value_user(struct kvm_vcpu *vcpu, u64 select_idx, u64 val);
u64 kvm_pmu_implemented_counter_mask(struct kvm_vcpu *vcpu);
+u64 kvm_pmu_hyp_counter_mask(struct kvm_vcpu *vcpu);
u64 kvm_pmu_accessible_counter_mask(struct kvm_vcpu *vcpu);
+u32 kvm_pmu_event_mask(struct kvm *kvm);
u64 kvm_pmu_get_pmceid(struct kvm_vcpu *vcpu, bool pmceid1);
void kvm_pmu_vcpu_init(struct kvm_vcpu *vcpu);
void kvm_pmu_vcpu_destroy(struct kvm_vcpu *vcpu);
void kvm_pmu_reprogram_counter_mask(struct kvm_vcpu *vcpu, u64 val);
void kvm_pmu_flush_hwstate(struct kvm_vcpu *vcpu);
void kvm_pmu_sync_hwstate(struct kvm_vcpu *vcpu);
+bool kvm_pmu_overflow_status(struct kvm_vcpu *vcpu);
bool kvm_pmu_should_notify_user(struct kvm_vcpu *vcpu);
bool kvm_pmu_update_run(struct kvm_vcpu *vcpu);
void kvm_pmu_software_increment(struct kvm_vcpu *vcpu, u64 val);
@@ -133,10 +137,20 @@ static inline u64 kvm_pmu_implemented_counter_mask(struct kvm_vcpu *vcpu)
{
return 0;
}
+
+static inline u64 kvm_pmu_hyp_counter_mask(struct kvm_vcpu *vcpu)
+{
+ return 0;
+}
+
static inline u64 kvm_pmu_accessible_counter_mask(struct kvm_vcpu *vcpu)
{
return 0;
}
+static inline u32 kvm_pmu_event_mask(struct kvm *kvm)
+{
+ return 0;
+}
static inline void kvm_pmu_vcpu_init(struct kvm_vcpu *vcpu) {}
static inline void kvm_pmu_vcpu_destroy(struct kvm_vcpu *vcpu) {}
static inline void kvm_pmu_reprogram_counter_mask(struct kvm_vcpu *vcpu, u64 val) {}
--
2.56.1.708.g77570a6a1d-goog
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v10 04/33] KVM: arm64: Use nr_pmu_counters in kvm_pmu_implemented_counter_mask()
2026-11-19 18:51 [PATCH v10 00/33] ARM64 PMU Partitioning Colton Lewis
` (2 preceding siblings ...)
2026-11-19 18:51 ` [PATCH v10 03/33] KVM: arm64: Reorganize PMU functions Colton Lewis
@ 2026-11-19 18:51 ` Colton Lewis
2026-11-19 19:08 ` sashiko-bot
2026-11-19 18:51 ` [PATCH v10 05/33] KVM: arm64: Ignore HPMN for userspace PMU register access Colton Lewis
` (28 subsequent siblings)
32 siblings, 1 reply; 67+ messages in thread
From: Colton Lewis @ 2026-11-19 18:51 UTC (permalink / raw)
To: Marc Zyngier, Oliver Upton
Cc: Catalin Marinas, Will Deacon, Mark Rutland, Peter Zijlstra,
Ingo Molnar, Paolo Bonzini, Russell King, Jonathan Corbet,
Fuad Tabba, Joey Gouly, Suzuki K Poulose, Zenghui Yu, Shuah Khan,
James Clark, Alexandru Elisei, Mingwei Zhang, Ganapatrao Kulkarni,
kvmarm, linux-arm-kernel, kvm, linux-perf-users, linux-kselftest,
linux-doc, linux-kernel
kvm_pmu_implemented_counter_mask() returns the bitmask of all counters
implemented by the VM (0..N-1 and PMCCNTR_EL0). Previously it extracted
N from kvm_vcpu_read_pmcr(vcpu), which commit 600f6fa5c90c ("KVM: arm64:
Let kvm_vcpu_read_pmcr() return an EL-dependent value for PMCR_EL0.N")
changed to substitute MDCR_EL2.HPMN for PMCR_EL0.N whenever
vcpu_has_nv(vcpu) && !vcpu_is_el2(vcpu).
As a result, when kvm_vcpu_reload_pmu() services KVM_REQ_RELOAD_PMU
while the vCPU is at vEL1 or vEL0 (for example, when entering a nested
guest after an L1 MDCR_EL2 update), kvm_pmu_implemented_counter_mask()
omits HPMN..N-1 and kvm_vcpu_reload_pmu() wipes L1's hypervisor counter
bits in PMOVSSET_EL0, PMINTENSET_EL1, and PMCNTENSET_EL0.
Fix this in three parts:
- Read vcpu->kvm->arch.nr_pmu_counters directly in
kvm_pmu_implemented_counter_mask() so the VM-wide implemented mask is
independent of the vCPU's current exception level.
- Use kvm_pmu_accessible_counter_mask(vcpu) & ~BIT(ARMV8_PMU_CYCLE_IDX)
for PMCR_EL0.P event counter resets instead of open-coding the
subtraction of kvm_pmu_hyp_counter_mask(vcpu) below EL2.
- Mask trapped reads in access_pmcnten(), access_pminten(), and
access_pmovs() with kvm_pmu_accessible_counter_mask(vcpu) (matching
existing write masking) so retained hypervisor counter bits in
HPMN..N-1 read as zero at EL1 and EL0.
Fixes: 600f6fa5c90c ("KVM: arm64: Let kvm_vcpu_read_pmcr() return an EL-dependent value for PMCR_EL0.N")
Fixes: 9a1c58cfefb0 ("KVM: arm64: nv: Adjust range of accessible PMCs according to HPMN")
Signed-off-by: Colton Lewis <coltonlewis@google.com>
---
arch/arm64/kvm/pmu-emul.c | 5 +----
arch/arm64/kvm/pmu.c | 2 +-
arch/arm64/kvm/sys_regs.c | 6 +++---
3 files changed, 5 insertions(+), 8 deletions(-)
diff --git a/arch/arm64/kvm/pmu-emul.c b/arch/arm64/kvm/pmu-emul.c
index ea5908df42890..9eff208547746 100644
--- a/arch/arm64/kvm/pmu-emul.c
+++ b/arch/arm64/kvm/pmu-emul.c
@@ -446,12 +446,9 @@ void kvm_pmu_handle_pmcr(struct kvm_vcpu *vcpu, u64 val)
kvm_pmu_set_counter_value(vcpu, ARMV8_PMU_CYCLE_IDX, 0);
if (val & ARMV8_PMU_PMCR_P) {
- unsigned long mask = kvm_pmu_implemented_counter_mask(vcpu) &
+ unsigned long mask = kvm_pmu_accessible_counter_mask(vcpu) &
~BIT(ARMV8_PMU_CYCLE_IDX);
- if (!vcpu_is_el2(vcpu))
- mask &= ~kvm_pmu_hyp_counter_mask(vcpu);
-
for_each_set_bit(i, &mask, 32)
kvm_pmu_set_pmc_value(kvm_vcpu_idx_to_pmc(vcpu, i), 0, true);
}
diff --git a/arch/arm64/kvm/pmu.c b/arch/arm64/kvm/pmu.c
index 567de460205a0..dbbe219ac874f 100644
--- a/arch/arm64/kvm/pmu.c
+++ b/arch/arm64/kvm/pmu.c
@@ -852,7 +852,7 @@ u8 kvm_arm_pmu_get_pmuver_limit(void)
u64 kvm_pmu_implemented_counter_mask(struct kvm_vcpu *vcpu)
{
- u64 val = FIELD_GET(ARMV8_PMU_PMCR_N, kvm_vcpu_read_pmcr(vcpu));
+ u64 val = vcpu->kvm->arch.nr_pmu_counters;
if (val == 0)
return BIT(ARMV8_PMU_CYCLE_IDX);
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 04947bfbd0ee5..6e8bc4618870d 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1359,7 +1359,7 @@ static bool access_pmcnten(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
kvm_pmu_reprogram_counter_mask(vcpu, val);
} else {
- p->regval = __vcpu_sys_reg(vcpu, PMCNTENSET_EL0);
+ p->regval = __vcpu_sys_reg(vcpu, PMCNTENSET_EL0) & mask;
}
return true;
@@ -1383,7 +1383,7 @@ static bool access_pminten(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
/* accessing PMINTENCLR_EL1 */
__vcpu_rmw_sys_reg(vcpu, PMINTENSET_EL1, &=, ~val);
} else {
- p->regval = __vcpu_sys_reg(vcpu, PMINTENSET_EL1);
+ p->regval = __vcpu_sys_reg(vcpu, PMINTENSET_EL1) & mask;
}
return true;
@@ -1463,7 +1463,7 @@ static bool access_pmovs(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
/* accessing PMOVSCLR_EL0 */
__vcpu_rmw_sys_reg(vcpu, PMOVSSET_EL0, &=, ~(p->regval & mask));
} else {
- p->regval = __vcpu_sys_reg(vcpu, PMOVSSET_EL0);
+ p->regval = __vcpu_sys_reg(vcpu, PMOVSSET_EL0) & mask;
}
return true;
--
2.56.1.708.g77570a6a1d-goog
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v10 05/33] KVM: arm64: Ignore HPMN for userspace PMU register access
2026-11-19 18:51 [PATCH v10 00/33] ARM64 PMU Partitioning Colton Lewis
` (3 preceding siblings ...)
2026-11-19 18:51 ` [PATCH v10 04/33] KVM: arm64: Use nr_pmu_counters in kvm_pmu_implemented_counter_mask() Colton Lewis
@ 2026-11-19 18:51 ` Colton Lewis
2026-11-19 18:58 ` sashiko-bot
2026-11-19 18:51 ` [PATCH v10 06/33] perf: arm_pmuv3: Generalize counter bitmasks Colton Lewis
` (27 subsequent siblings)
32 siblings, 1 reply; 67+ messages in thread
From: Colton Lewis @ 2026-11-19 18:51 UTC (permalink / raw)
To: Marc Zyngier, Oliver Upton
Cc: Catalin Marinas, Will Deacon, Mark Rutland, Peter Zijlstra,
Ingo Molnar, Paolo Bonzini, Russell King, Jonathan Corbet,
Fuad Tabba, Joey Gouly, Suzuki K Poulose, Zenghui Yu, Shuah Khan,
James Clark, Alexandru Elisei, Mingwei Zhang, Ganapatrao Kulkarni,
kvmarm, linux-arm-kernel, kvm, linux-perf-users, linux-kselftest,
linux-doc, linux-kernel
Userspace save and restore (KVM_GET_ONE_REG / KVM_SET_ONE_REG) must
observe and preserve all VM-implemented PMU counters
([0..nr_pmu_counters-1] plus PMCCNTR_EL0), independent of the vCPU's
current exception level or MDCR_EL2.HPMN.
Currently, two UAPI callbacks still depend on guest execution context
when a nested vCPU (KVM_ARM_VCPU_HAS_EL2) is paused below EL2
(!vcpu_is_el2(vcpu)) with MDCR_EL2.HPMN < nr_pmu_counters:
- set_pmreg() and get_pmreg() (backing PMINTEN{SET,CLR}_EL1,
PMCNTEN{SET,CLR}_EL0, and PMOVS{SET,CLR}_EL0) mask register values
with kvm_pmu_accessible_counter_mask(), which strips L1 hypervisor
counter bits [HPMN..N-1]. Saving a vCPU paused in L2 hides those bits,
and restoring them while paused in L2 permanently clears them.
- get_pmcr() calls kvm_vcpu_read_pmcr(), which substitutes
MDCR_EL2.HPMN for PMCR_EL0.N below EL2, corrupting the saved
PMCR_EL0.N value.
Switch set_pmreg() and get_pmreg() back to
kvm_pmu_implemented_counter_mask(), and have get_pmcr() populate
PMCR_EL0.N directly from vcpu->kvm->arch.nr_pmu_counters using
u64_replace_bits(). Keep kvm_vcpu_read_pmcr() and
kvm_pmu_accessible_counter_mask() for guest trap paths.
Fixes: 9a1c58cfefb0 ("KVM: arm64: nv: Adjust range of accessible PMCs according to HPMN")
Fixes: 600f6fa5c90c ("KVM: arm64: Let kvm_vcpu_read_pmcr() return an EL-dependent value for PMCR_EL0.N")
Signed-off-by: Colton Lewis <coltonlewis@google.com>
---
arch/arm64/kvm/sys_regs.c | 12 +++++++-----
1 file changed, 7 insertions(+), 5 deletions(-)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 6e8bc4618870d..2d9ae781acff0 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1081,8 +1081,8 @@ static u64 reset_pmcr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r)
pmcr |= ARMV8_PMU_PMCR_LC;
/*
- * The value of PMCR.N field is included when the
- * vCPU register is read via kvm_vcpu_read_pmcr().
+ * The value of PMCR.N field is included when the vCPU register
+ * is read via kvm_vcpu_read_pmcr() or get_pmcr().
*/
__vcpu_assign_sys_reg(vcpu, r->reg, pmcr);
@@ -1323,7 +1323,7 @@ static bool access_pmu_evtyper(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
static int set_pmreg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r, u64 val)
{
- u64 mask = kvm_pmu_accessible_counter_mask(vcpu);
+ u64 mask = kvm_pmu_implemented_counter_mask(vcpu);
__vcpu_assign_sys_reg(vcpu, r->reg, val & mask);
kvm_make_request(KVM_REQ_RELOAD_PMU, vcpu);
@@ -1333,7 +1333,7 @@ static int set_pmreg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r, u64 va
static int get_pmreg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r, u64 *val)
{
- u64 mask = kvm_pmu_accessible_counter_mask(vcpu);
+ u64 mask = kvm_pmu_implemented_counter_mask(vcpu);
*val = __vcpu_sys_reg(vcpu, r->reg) & mask;
return 0;
@@ -1505,7 +1505,9 @@ static bool access_pmuserenr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
static int get_pmcr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r,
u64 *val)
{
- *val = kvm_vcpu_read_pmcr(vcpu);
+ *val = u64_replace_bits(__vcpu_sys_reg(vcpu, PMCR_EL0),
+ vcpu->kvm->arch.nr_pmu_counters,
+ ARMV8_PMU_PMCR_N);
return 0;
}
--
2.56.1.708.g77570a6a1d-goog
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v10 06/33] perf: arm_pmuv3: Generalize counter bitmasks
2026-11-19 18:51 [PATCH v10 00/33] ARM64 PMU Partitioning Colton Lewis
` (4 preceding siblings ...)
2026-11-19 18:51 ` [PATCH v10 05/33] KVM: arm64: Ignore HPMN for userspace PMU register access Colton Lewis
@ 2026-11-19 18:51 ` Colton Lewis
2026-11-19 18:56 ` sashiko-bot
2026-11-19 18:51 ` [PATCH v10 07/33] perf: arm_pmuv3: Move counter allocation mask to per-CPU struct pmu_hw_events Colton Lewis
` (26 subsequent siblings)
32 siblings, 1 reply; 67+ messages in thread
From: Colton Lewis @ 2026-11-19 18:51 UTC (permalink / raw)
To: Marc Zyngier, Oliver Upton
Cc: Catalin Marinas, Will Deacon, Mark Rutland, Peter Zijlstra,
Ingo Molnar, Paolo Bonzini, Russell King, Jonathan Corbet,
Fuad Tabba, Joey Gouly, Suzuki K Poulose, Zenghui Yu, Shuah Khan,
James Clark, Alexandru Elisei, Mingwei Zhang, Ganapatrao Kulkarni,
kvmarm, linux-arm-kernel, kvm, linux-perf-users, linux-kselftest,
linux-doc, linux-kernel
The OVSR bitmasks are valid for enable and interrupt registers as well as
overflow registers. Generalize the names.
Acked-by: Mark Rutland <mark.rutland@arm.com>
Tested-by: James Clark <james.clark@linaro.org>
Signed-off-by: Colton Lewis <coltonlewis@google.com>
---
drivers/perf/arm_pmuv3.c | 4 ++--
include/linux/perf/arm_pmuv3.h | 14 +++++++-------
tools/include/perf/arm_pmuv3.h | 12 +++++++-----
3 files changed, 16 insertions(+), 14 deletions(-)
diff --git a/drivers/perf/arm_pmuv3.c b/drivers/perf/arm_pmuv3.c
index fbd631ec71a65..4a6c1f3bcea1f 100644
--- a/drivers/perf/arm_pmuv3.c
+++ b/drivers/perf/arm_pmuv3.c
@@ -535,7 +535,7 @@ static void armv8pmu_pmcr_write(u64 val)
static int armv8pmu_has_overflowed(u64 pmovsr)
{
- return !!(pmovsr & ARMV8_PMU_OVERFLOWED_MASK);
+ return !!(pmovsr & ARMV8_PMU_CNT_MASK_ALL);
}
static int armv8pmu_counter_has_overflowed(u64 pmnc, int idx)
@@ -771,7 +771,7 @@ static u64 armv8pmu_getreset_flags(void)
value = read_pmovsclr();
/* Write to clear flags */
- value &= ARMV8_PMU_OVERFLOWED_MASK;
+ value &= ARMV8_PMU_CNT_MASK_ALL;
write_pmovsclr(value);
return value;
diff --git a/include/linux/perf/arm_pmuv3.h b/include/linux/perf/arm_pmuv3.h
index d698efba28a27..fd2a34b4a64d1 100644
--- a/include/linux/perf/arm_pmuv3.h
+++ b/include/linux/perf/arm_pmuv3.h
@@ -224,14 +224,14 @@
ARMV8_PMU_PMCR_LC | ARMV8_PMU_PMCR_LP)
/*
- * PMOVSR: counters overflow flag status reg
+ * Counter bitmask layouts for overflow, enable, and interrupts
*/
-#define ARMV8_PMU_OVSR_P GENMASK(30, 0)
-#define ARMV8_PMU_OVSR_C BIT(31)
-#define ARMV8_PMU_OVSR_F BIT_ULL(32) /* arm64 only */
-/* Mask for writable bits is both P and C fields */
-#define ARMV8_PMU_OVERFLOWED_MASK (ARMV8_PMU_OVSR_P | ARMV8_PMU_OVSR_C | \
- ARMV8_PMU_OVSR_F)
+#define ARMV8_PMU_CNT_MASK_P GENMASK(30, 0)
+#define ARMV8_PMU_CNT_MASK_C BIT(31)
+#define ARMV8_PMU_CNT_MASK_F BIT_ULL(32) /* arm64 only */
+#define ARMV8_PMU_CNT_MASK_ALL (ARMV8_PMU_CNT_MASK_P | \
+ ARMV8_PMU_CNT_MASK_C | \
+ ARMV8_PMU_CNT_MASK_F)
/*
* PMXEVTYPER: Event selection reg
diff --git a/tools/include/perf/arm_pmuv3.h b/tools/include/perf/arm_pmuv3.h
index 1e397d55384ed..d045b0f3b93fe 100644
--- a/tools/include/perf/arm_pmuv3.h
+++ b/tools/include/perf/arm_pmuv3.h
@@ -226,12 +226,14 @@
ARMV8_PMU_PMCR_LC | ARMV8_PMU_PMCR_LP)
/*
- * PMOVSR: counters overflow flag status reg
+ * Counter bitmask layouts for overflow, enable, and interrupts
*/
-#define ARMV8_PMU_OVSR_P GENMASK(30, 0)
-#define ARMV8_PMU_OVSR_C BIT(31)
-/* Mask for writable bits is both P and C fields */
-#define ARMV8_PMU_OVERFLOWED_MASK (ARMV8_PMU_OVSR_P | ARMV8_PMU_OVSR_C)
+#define ARMV8_PMU_CNT_MASK_P GENMASK(30, 0)
+#define ARMV8_PMU_CNT_MASK_C BIT(31)
+#define ARMV8_PMU_CNT_MASK_F BIT_ULL(32) /* arm64 only */
+#define ARMV8_PMU_CNT_MASK_ALL (ARMV8_PMU_CNT_MASK_P | \
+ ARMV8_PMU_CNT_MASK_C | \
+ ARMV8_PMU_CNT_MASK_F)
/*
* PMXEVTYPER: Event selection reg
--
2.56.1.708.g77570a6a1d-goog
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v10 07/33] perf: arm_pmuv3: Move counter allocation mask to per-CPU struct pmu_hw_events
2026-11-19 18:51 [PATCH v10 00/33] ARM64 PMU Partitioning Colton Lewis
` (5 preceding siblings ...)
2026-11-19 18:51 ` [PATCH v10 06/33] perf: arm_pmuv3: Generalize counter bitmasks Colton Lewis
@ 2026-11-19 18:51 ` Colton Lewis
2026-11-19 19:04 ` sashiko-bot
2026-11-19 18:51 ` [PATCH v10 08/33] perf: arm_pmuv3: Check cntr_mask before using pmccntr Colton Lewis
` (25 subsequent siblings)
32 siblings, 1 reply; 67+ messages in thread
From: Colton Lewis @ 2026-11-19 18:51 UTC (permalink / raw)
To: Marc Zyngier, Oliver Upton
Cc: Catalin Marinas, Will Deacon, Mark Rutland, Peter Zijlstra,
Ingo Molnar, Paolo Bonzini, Russell King, Jonathan Corbet,
Fuad Tabba, Joey Gouly, Suzuki K Poulose, Zenghui Yu, Shuah Khan,
James Clark, Alexandru Elisei, Mingwei Zhang, Ganapatrao Kulkarni,
kvmarm, linux-arm-kernel, kvm, linux-perf-users, linux-kselftest,
linux-doc, linux-kernel
In preparation for dynamic per-CPU PMU counter reservations when running
KVM guests with a partitioned PMU, move the counter allocation mask from
the global struct arm_pmu to per-CPU struct pmu_hw_events.
Initialize cpuc->cntr_mask from the static cpu_pmu->cntr_mask during PMU
probe and CPU hotplug startup, and update event counter allocation
helpers (armv8pmu_get_single_idx(), armv8pmu_get_chain_idx(), and
armv8pmu_get_event_idx()) to query the per-CPU mask.
Tested-by: James Clark <james.clark@linaro.org>
Signed-off-by: Colton Lewis <coltonlewis@google.com>
---
drivers/perf/arm_pmu.c | 9 +++++++--
drivers/perf/arm_pmuv3.c | 28 +++++++++++++++++-----------
include/linux/perf/arm_pmu.h | 1 +
3 files changed, 25 insertions(+), 13 deletions(-)
diff --git a/drivers/perf/arm_pmu.c b/drivers/perf/arm_pmu.c
index 1150695653892..b1851d1c594b1 100644
--- a/drivers/perf/arm_pmu.c
+++ b/drivers/perf/arm_pmu.c
@@ -408,9 +408,11 @@ validate_group(struct perf_event *event)
/*
* Initialise the fake PMU. We only need to populate the
- * used_mask for the purposes of validation.
+ * used_mask and cntr_mask for the purposes of validation.
*/
memset(&fake_pmu.used_mask, 0, sizeof(fake_pmu.used_mask));
+ bitmap_copy(fake_pmu.cntr_mask, to_arm_pmu(event->pmu)->cntr_mask,
+ ARMPMU_MAX_HWEVENTS);
if (!validate_event(event->pmu, &fake_pmu, leader))
return -EINVAL;
@@ -717,6 +719,7 @@ bool arm_pmu_irq_is_nmi(void)
static int arm_perf_starting_cpu(unsigned int cpu, struct hlist_node *node)
{
struct arm_pmu *pmu = hlist_entry_safe(node, struct arm_pmu, node);
+ struct pmu_hw_events *cpuc = per_cpu_ptr(pmu->hw_events, cpu);
int irq;
if (!cpumask_test_cpu(cpu, &pmu->supported_cpus))
@@ -724,6 +727,8 @@ static int arm_perf_starting_cpu(unsigned int cpu, struct hlist_node *node)
if (pmu->reset)
pmu->reset(pmu);
+ bitmap_copy(cpuc->cntr_mask, pmu->cntr_mask, ARMPMU_MAX_HWEVENTS);
+
irq = armpmu_get_cpu_irq(pmu, cpu);
if (irq)
per_cpu(cpu_irq_ops, cpu)->enable_pmuirq(irq);
@@ -753,7 +758,7 @@ static void cpu_pm_pmu_setup(struct arm_pmu *armpmu, unsigned long cmd)
struct perf_event *event;
int idx;
- for_each_set_bit(idx, armpmu->cntr_mask, ARMPMU_MAX_HWEVENTS) {
+ for_each_set_bit(idx, hw_events->cntr_mask, ARMPMU_MAX_HWEVENTS) {
event = hw_events->events[idx];
if (!event)
continue;
diff --git a/drivers/perf/arm_pmuv3.c b/drivers/perf/arm_pmuv3.c
index 4a6c1f3bcea1f..973d4abcb3a6e 100644
--- a/drivers/perf/arm_pmuv3.c
+++ b/drivers/perf/arm_pmuv3.c
@@ -813,7 +813,7 @@ static void armv8pmu_enable_user_access(struct arm_pmu *cpu_pmu)
write_pmuacr(mask);
} else {
/* Clear any unused counters to avoid leaking their contents */
- for_each_andnot_bit(i, cpu_pmu->cntr_mask, cpuc->used_mask,
+ for_each_andnot_bit(i, cpuc->cntr_mask, cpuc->used_mask,
ARMPMU_MAX_HWEVENTS) {
if (i == ARMV8_PMU_CYCLE_IDX)
write_pmccntr(0);
@@ -917,7 +917,7 @@ static irqreturn_t armv8pmu_handle_irq(struct arm_pmu *cpu_pmu)
* to prevent skews in group events.
*/
armv8pmu_stop(cpu_pmu);
- for_each_set_bit(idx, cpu_pmu->cntr_mask, ARMPMU_MAX_HWEVENTS) {
+ for_each_set_bit(idx, cpuc->cntr_mask, ARMPMU_MAX_HWEVENTS) {
struct perf_event *event = cpuc->events[idx];
struct hw_perf_event *hwc;
@@ -953,20 +953,18 @@ static irqreturn_t armv8pmu_handle_irq(struct arm_pmu *cpu_pmu)
return IRQ_HANDLED;
}
-static int armv8pmu_get_single_idx(struct pmu_hw_events *cpuc,
- struct arm_pmu *cpu_pmu)
+static int armv8pmu_get_single_idx(struct pmu_hw_events *cpuc)
{
int idx;
- for_each_set_bit(idx, cpu_pmu->cntr_mask, ARMV8_PMU_MAX_GENERAL_COUNTERS) {
+ for_each_set_bit(idx, cpuc->cntr_mask, ARMV8_PMU_MAX_GENERAL_COUNTERS) {
if (!test_and_set_bit(idx, cpuc->used_mask))
return idx;
}
return -EAGAIN;
}
-static int armv8pmu_get_chain_idx(struct pmu_hw_events *cpuc,
- struct arm_pmu *cpu_pmu)
+static int armv8pmu_get_chain_idx(struct pmu_hw_events *cpuc)
{
int idx;
@@ -974,7 +972,7 @@ static int armv8pmu_get_chain_idx(struct pmu_hw_events *cpuc,
* Chaining requires two consecutive event counters, where
* the lower idx must be even.
*/
- for_each_set_bit(idx, cpu_pmu->cntr_mask, ARMV8_PMU_MAX_GENERAL_COUNTERS) {
+ for_each_set_bit(idx, cpuc->cntr_mask, ARMV8_PMU_MAX_GENERAL_COUNTERS) {
if (!(idx & 0x1))
continue;
if (!test_and_set_bit(idx, cpuc->used_mask)) {
@@ -1042,7 +1040,7 @@ static int armv8pmu_get_event_idx(struct pmu_hw_events *cpuc,
*/
if ((evtype == ARMV8_PMUV3_PERFCTR_INST_RETIRED) &&
!armv8pmu_event_get_threshold(&event->attr) &&
- test_bit(ARMV8_PMU_INSTR_IDX, cpu_pmu->cntr_mask) &&
+ test_bit(ARMV8_PMU_INSTR_IDX, cpuc->cntr_mask) &&
!armv8pmu_event_want_user_access(event)) {
if (!test_and_set_bit(ARMV8_PMU_INSTR_IDX, cpuc->used_mask))
return ARMV8_PMU_INSTR_IDX;
@@ -1052,9 +1050,9 @@ static int armv8pmu_get_event_idx(struct pmu_hw_events *cpuc,
* Otherwise use events counters
*/
if (armv8pmu_event_is_chained(event))
- return armv8pmu_get_chain_idx(cpuc, cpu_pmu);
+ return armv8pmu_get_chain_idx(cpuc);
else
- return armv8pmu_get_single_idx(cpuc, cpu_pmu);
+ return armv8pmu_get_single_idx(cpuc);
}
static void armv8pmu_clear_event_idx(struct pmu_hw_events *cpuc,
@@ -1426,6 +1424,7 @@ static int armv8pmu_probe_pmu(struct arm_pmu *cpu_pmu)
.present = false,
};
int ret;
+ int cpu;
ret = smp_call_function_any(&cpu_pmu->supported_cpus,
__armv8pmu_probe_pmu,
@@ -1441,6 +1440,13 @@ static int armv8pmu_probe_pmu(struct arm_pmu *cpu_pmu)
if (ret)
return ret;
}
+
+ for_each_possible_cpu(cpu) {
+ struct pmu_hw_events *cpuc = per_cpu_ptr(cpu_pmu->hw_events, cpu);
+
+ bitmap_copy(cpuc->cntr_mask, cpu_pmu->cntr_mask, ARMPMU_MAX_HWEVENTS);
+ }
+
return 0;
}
diff --git a/include/linux/perf/arm_pmu.h b/include/linux/perf/arm_pmu.h
index 02d2c7f45b527..be1e345e99a77 100644
--- a/include/linux/perf/arm_pmu.h
+++ b/include/linux/perf/arm_pmu.h
@@ -62,6 +62,7 @@ struct pmu_hw_events {
* an event. A 0 means that the counter can be used.
*/
DECLARE_BITMAP(used_mask, ARMPMU_MAX_HWEVENTS);
+ DECLARE_BITMAP(cntr_mask, ARMPMU_MAX_HWEVENTS);
/*
* When using percpu IRQs, we need a percpu dev_id. Place it here as we
--
2.56.1.708.g77570a6a1d-goog
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v10 08/33] perf: arm_pmuv3: Check cntr_mask before using pmccntr
2026-11-19 18:51 [PATCH v10 00/33] ARM64 PMU Partitioning Colton Lewis
` (6 preceding siblings ...)
2026-11-19 18:51 ` [PATCH v10 07/33] perf: arm_pmuv3: Move counter allocation mask to per-CPU struct pmu_hw_events Colton Lewis
@ 2026-11-19 18:51 ` Colton Lewis
2026-11-19 19:01 ` sashiko-bot
2026-11-19 18:51 ` [PATCH v10 09/33] perf: arm_pmuv3: Allocate counter indices from high to low Colton Lewis
` (24 subsequent siblings)
32 siblings, 1 reply; 67+ messages in thread
From: Colton Lewis @ 2026-11-19 18:51 UTC (permalink / raw)
To: Marc Zyngier, Oliver Upton
Cc: Catalin Marinas, Will Deacon, Mark Rutland, Peter Zijlstra,
Ingo Molnar, Paolo Bonzini, Russell King, Jonathan Corbet,
Fuad Tabba, Joey Gouly, Suzuki K Poulose, Zenghui Yu, Shuah Khan,
James Clark, Alexandru Elisei, Mingwei Zhang, Ganapatrao Kulkarni,
kvmarm, linux-arm-kernel, kvm, linux-perf-users, linux-kselftest,
linux-doc, linux-kernel
When the PMU is partitioned, PMCCNTR_EL0 is reserved for the guest and
removed from cpuc->cntr_mask. Move the PMCCNTR_EL0 availability checks
(cpuc->cntr_mask and cpuc->used_mask) into armv8pmu_can_use_pmccntr()
and un-nest the 64-bit user-access fallback check in
armv8pmu_get_event_idx() so host CPU_CYCLES events fall back to
general-purpose event counters when PMCCNTR_EL0 is unavailable.
Tested-by: James Clark <james.clark@linaro.org>
Signed-off-by: Colton Lewis <coltonlewis@google.com>
---
drivers/perf/arm_pmuv3.c | 17 +++++++++++------
1 file changed, 11 insertions(+), 6 deletions(-)
diff --git a/drivers/perf/arm_pmuv3.c b/drivers/perf/arm_pmuv3.c
index 973d4abcb3a6e..9ce915754b0a5 100644
--- a/drivers/perf/arm_pmuv3.c
+++ b/drivers/perf/arm_pmuv3.c
@@ -1013,6 +1013,10 @@ static bool armv8pmu_can_use_pmccntr(struct pmu_hw_events *cpuc,
if (cpu_pmu->avoid_pmccntr)
return false;
+ if (!test_bit(ARMV8_PMU_CYCLE_IDX, cpuc->cntr_mask) ||
+ test_bit(ARMV8_PMU_CYCLE_IDX, cpuc->used_mask))
+ return false;
+
return true;
}
@@ -1025,14 +1029,15 @@ static int armv8pmu_get_event_idx(struct pmu_hw_events *cpuc,
/* Always prefer to place a cycle counter into the cycle counter. */
if (armv8pmu_can_use_pmccntr(cpuc, event)) {
- if (!test_and_set_bit(ARMV8_PMU_CYCLE_IDX, cpuc->used_mask))
- return ARMV8_PMU_CYCLE_IDX;
- else if (armv8pmu_event_is_64bit(event) &&
- armv8pmu_event_want_user_access(event) &&
- !armv8pmu_has_long_event(cpu_pmu))
- return -EAGAIN;
+ set_bit(ARMV8_PMU_CYCLE_IDX, cpuc->used_mask);
+ return ARMV8_PMU_CYCLE_IDX;
}
+ if (armv8pmu_event_is_64bit(event) &&
+ armv8pmu_event_want_user_access(event) &&
+ !armv8pmu_has_long_event(cpu_pmu))
+ return -EAGAIN;
+
/*
* Always prefer to place a instruction counter into the instruction counter,
* but don't expose the instruction counter to userspace access as userspace
--
2.56.1.708.g77570a6a1d-goog
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v10 09/33] perf: arm_pmuv3: Allocate counter indices from high to low
2026-11-19 18:51 [PATCH v10 00/33] ARM64 PMU Partitioning Colton Lewis
` (7 preceding siblings ...)
2026-11-19 18:51 ` [PATCH v10 08/33] perf: arm_pmuv3: Check cntr_mask before using pmccntr Colton Lewis
@ 2026-11-19 18:51 ` Colton Lewis
2026-11-19 18:57 ` sashiko-bot
2026-11-19 18:51 ` [PATCH v10 10/33] perf: arm_pmuv3: Expose number of general-purpose counters in sysfs Colton Lewis
` (23 subsequent siblings)
32 siblings, 1 reply; 67+ messages in thread
From: Colton Lewis @ 2026-11-19 18:51 UTC (permalink / raw)
To: Marc Zyngier, Oliver Upton
Cc: Catalin Marinas, Will Deacon, Mark Rutland, Peter Zijlstra,
Ingo Molnar, Paolo Bonzini, Russell King, Jonathan Corbet,
Fuad Tabba, Joey Gouly, Suzuki K Poulose, Zenghui Yu, Shuah Khan,
James Clark, Alexandru Elisei, Mingwei Zhang, Ganapatrao Kulkarni,
kvmarm, linux-arm-kernel, kvm, linux-perf-users, linux-kselftest,
linux-doc, linux-kernel
MDCR_EL2.HPMN assigns low-index general-purpose counters (0..HPMN-1) to
the guest and high-index counters (HPMN..N-1) to the host. Allocate host
event counters from high index to low index to minimize collisions with
guest counter partitions and avoid rescheduling host perf events on vCPU
load when enough upper counters are free.
When allocating a 64-bit chained pair (even counter idx - 1 paired with
odd counter idx), verify that both idx and idx - 1 are present in
cpuc->cntr_mask so an odd HPMN boundary cannot allocate a guest-owned
even counter as the lower half of a host chained event.
Tested-by: James Clark <james.clark@linaro.org>
Signed-off-by: Colton Lewis <coltonlewis@google.com>
---
drivers/perf/arm_pmuv3.c | 25 ++++++++++++++++---------
1 file changed, 16 insertions(+), 9 deletions(-)
diff --git a/drivers/perf/arm_pmuv3.c b/drivers/perf/arm_pmuv3.c
index 9ce915754b0a5..667ce27d2347c 100644
--- a/drivers/perf/arm_pmuv3.c
+++ b/drivers/perf/arm_pmuv3.c
@@ -957,10 +957,12 @@ static int armv8pmu_get_single_idx(struct pmu_hw_events *cpuc)
{
int idx;
- for_each_set_bit(idx, cpuc->cntr_mask, ARMV8_PMU_MAX_GENERAL_COUNTERS) {
- if (!test_and_set_bit(idx, cpuc->used_mask))
+ for (idx = ARMV8_PMU_MAX_GENERAL_COUNTERS - 1; idx >= 0; idx--) {
+ if (test_bit(idx, cpuc->cntr_mask) &&
+ !test_and_set_bit(idx, cpuc->used_mask))
return idx;
}
+
return -EAGAIN;
}
@@ -972,17 +974,22 @@ static int armv8pmu_get_chain_idx(struct pmu_hw_events *cpuc)
* Chaining requires two consecutive event counters, where
* the lower idx must be even.
*/
- for_each_set_bit(idx, cpuc->cntr_mask, ARMV8_PMU_MAX_GENERAL_COUNTERS) {
+ for (idx = ARMV8_PMU_MAX_GENERAL_COUNTERS - 1; idx >= 0; idx--) {
if (!(idx & 0x1))
continue;
- if (!test_and_set_bit(idx, cpuc->used_mask)) {
- /* Check if the preceding even counter is available */
- if (!test_and_set_bit(idx - 1, cpuc->used_mask))
- return idx;
- /* Release the Odd counter */
- clear_bit(idx, cpuc->used_mask);
+
+ if (test_bit(idx, cpuc->cntr_mask) &&
+ test_bit(idx - 1, cpuc->cntr_mask)) {
+ if (!test_and_set_bit(idx, cpuc->used_mask)) {
+ /* Check if the preceding even counter is available */
+ if (!test_and_set_bit(idx - 1, cpuc->used_mask))
+ return idx;
+ /* Release the Odd counter */
+ clear_bit(idx, cpuc->used_mask);
+ }
}
}
+
return -EAGAIN;
}
--
2.56.1.708.g77570a6a1d-goog
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v10 10/33] perf: arm_pmuv3: Expose number of general-purpose counters in sysfs
2026-11-19 18:51 [PATCH v10 00/33] ARM64 PMU Partitioning Colton Lewis
` (8 preceding siblings ...)
2026-11-19 18:51 ` [PATCH v10 09/33] perf: arm_pmuv3: Allocate counter indices from high to low Colton Lewis
@ 2026-11-19 18:51 ` Colton Lewis
2026-11-19 18:57 ` sashiko-bot
2026-11-19 18:51 ` [PATCH v10 11/33] KVM: arm64: Add initial scaffolding for Partitioned PMU Colton Lewis
` (22 subsequent siblings)
32 siblings, 1 reply; 67+ messages in thread
From: Colton Lewis @ 2026-11-19 18:51 UTC (permalink / raw)
To: Marc Zyngier, Oliver Upton
Cc: Catalin Marinas, Will Deacon, Mark Rutland, Peter Zijlstra,
Ingo Molnar, Paolo Bonzini, Russell King, Jonathan Corbet,
Fuad Tabba, Joey Gouly, Suzuki K Poulose, Zenghui Yu, Shuah Khan,
James Clark, Alexandru Elisei, Mingwei Zhang, Ganapatrao Kulkarni,
kvmarm, linux-arm-kernel, kvm, linux-perf-users, linux-kselftest,
linux-doc, linux-kernel
Expose the number of general-purpose event counters implemented by each
ARMv8 PMUv3 instance at
/sys/bus/event_source/devices/<pmu>/caps/num_counters.
This allows userspace (such as VMMs configuring
KVM_ARM_VCPU_PMU_V3_SET_NR_COUNTERS for a Partitioned PMU on
heterogeneous systems) to query the maximum number of general-purpose
counters for a specific PMU without reading PMCR_EL0.N directly from
EL0.
Suggested-by: James Clark <james.clark@linaro.org>
Signed-off-by: Colton Lewis <coltonlewis@google.com>
---
drivers/perf/arm_pmuv3.c | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/drivers/perf/arm_pmuv3.c b/drivers/perf/arm_pmuv3.c
index 667ce27d2347c..1676f13cc7c38 100644
--- a/drivers/perf/arm_pmuv3.c
+++ b/drivers/perf/arm_pmuv3.c
@@ -457,12 +457,26 @@ static ssize_t branches_show(struct device *dev,
static DEVICE_ATTR_RO(branches);
+static ssize_t num_counters_show(struct device *dev,
+ struct device_attribute *attr, char *page)
+{
+ struct pmu *pmu = dev_get_drvdata(dev);
+ struct arm_pmu *cpu_pmu = container_of(pmu, struct arm_pmu, pmu);
+ int nr_cntrs = bitmap_weight(cpu_pmu->cntr_mask,
+ ARMV8_PMU_MAX_GENERAL_COUNTERS);
+
+ return sysfs_emit(page, "%d\n", nr_cntrs);
+}
+
+static DEVICE_ATTR_RO(num_counters);
+
static struct attribute *armv8_pmuv3_caps_attrs[] = {
&dev_attr_branches.attr,
&dev_attr_slots.attr,
&dev_attr_bus_slots.attr,
&dev_attr_bus_width.attr,
&dev_attr_threshold_max.attr,
+ &dev_attr_num_counters.attr,
NULL,
};
--
2.56.1.708.g77570a6a1d-goog
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v10 11/33] KVM: arm64: Add initial scaffolding for Partitioned PMU
2026-11-19 18:51 [PATCH v10 00/33] ARM64 PMU Partitioning Colton Lewis
` (9 preceding siblings ...)
2026-11-19 18:51 ` [PATCH v10 10/33] perf: arm_pmuv3: Expose number of general-purpose counters in sysfs Colton Lewis
@ 2026-11-19 18:51 ` Colton Lewis
2026-11-19 18:58 ` sashiko-bot
2026-11-19 18:51 ` [PATCH v10 12/33] KVM: arm64: Set up FGT " Colton Lewis
` (21 subsequent siblings)
32 siblings, 1 reply; 67+ messages in thread
From: Colton Lewis @ 2026-11-19 18:51 UTC (permalink / raw)
To: Marc Zyngier, Oliver Upton
Cc: Catalin Marinas, Will Deacon, Mark Rutland, Peter Zijlstra,
Ingo Molnar, Paolo Bonzini, Russell King, Jonathan Corbet,
Fuad Tabba, Joey Gouly, Suzuki K Poulose, Zenghui Yu, Shuah Khan,
James Clark, Alexandru Elisei, Mingwei Zhang, Ganapatrao Kulkarni,
kvmarm, linux-arm-kernel, kvm, linux-perf-users, linux-kselftest,
linux-doc, linux-kernel
Add the foundational helpers for Partitioned PMU support in
arch/arm64/kvm/pmu-direct.c:
- has_kvm_pmu_partition_support() to check VHE and PMUv3 availability.
- kvm_pmu_partition_enable() and kvm_pmu_is_partitioned() to track per-VM
partitioning state via KVM_ARCH_FLAG_PARTITION_PMU_ENABLED.
- kvm_pmu_guest_counter_mask() to compute the bitmask of guest-reserved
(0..HPMN-1 and PMCCNTR_EL0) hardware counters.
Tested-by: James Clark <james.clark@linaro.org>
Signed-off-by: Colton Lewis <coltonlewis@google.com>
---
arch/arm/include/asm/arm_pmuv3.h | 4 ++
arch/arm64/include/asm/kvm_host.h | 2 +
arch/arm64/kvm/Makefile | 2 +-
arch/arm64/kvm/pmu-direct.c | 94 +++++++++++++++++++++++++++++++
include/kvm/arm_pmu.h | 20 +++++++
5 files changed, 121 insertions(+), 1 deletion(-)
create mode 100644 arch/arm64/kvm/pmu-direct.c
diff --git a/arch/arm/include/asm/arm_pmuv3.h b/arch/arm/include/asm/arm_pmuv3.h
index ecfede0c03486..71cf03ab58cbb 100644
--- a/arch/arm/include/asm/arm_pmuv3.h
+++ b/arch/arm/include/asm/arm_pmuv3.h
@@ -227,6 +227,10 @@ static inline bool kvm_set_pmuserenr(u64 val)
}
static inline void kvm_vcpu_pmu_resync_el0(void) {}
+static inline u64 kvm_pmu_guest_counter_mask(void)
+{
+ return 0;
+}
/* PMU Version in DFR Register */
#define ARMV8_PMU_DFR_VER_NI 0
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 4683281fcc5d4..89683a690828e 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -96,6 +96,8 @@ enum {
KVM_ARCH_FLAG_EXIT_SEA,
/* No further ID register changes possible */
KVM_ARCH_FLAG_ID_REGS_FINAL,
+ /* Partitioned PMU Enabled */
+ KVM_ARCH_FLAG_PARTITION_PMU_ENABLED,
};
struct vcpu_reset_state {
diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
index 7aaeeb84e788e..79f1602299767 100644
--- a/arch/arm64/kvm/Makefile
+++ b/arch/arm64/kvm/Makefile
@@ -27,7 +27,7 @@ kvm-y += arm.o mmu.o mmio.o psci.o hypercalls.o pvtime.o \
vgic/vgic-v5.o vgic/vgic-v5-tables.o vgic/vgic-irs-v5.o \
vgic-sys-reg-v5.o
-kvm-$(CONFIG_HW_PERF_EVENTS) += pmu-emul.o pmu.o
+kvm-$(CONFIG_HW_PERF_EVENTS) += pmu-emul.o pmu-direct.o pmu.o
kvm-$(CONFIG_ARM64_PTR_AUTH) += pauth.o
kvm-$(CONFIG_PTDUMP_STAGE2_DEBUGFS) += ptdump.o
diff --git a/arch/arm64/kvm/pmu-direct.c b/arch/arm64/kvm/pmu-direct.c
new file mode 100644
index 0000000000000..b1c180ae79d9d
--- /dev/null
+++ b/arch/arm64/kvm/pmu-direct.c
@@ -0,0 +1,94 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2025 Google LLC
+ * Author: Colton Lewis <coltonlewis@google.com>
+ */
+
+#include <linux/kvm_host.h>
+#include <linux/perf/arm_pmu.h>
+#include <linux/perf/arm_pmuv3.h>
+
+#include <asm/arm_pmuv3.h>
+#include <asm/kvm_emulate.h>
+#include <asm/virt.h>
+
+/**
+ * has_host_pmu_partition_support() - Determine if partitioning is possible
+ *
+ * Partitioning is only supported in VHE mode with PMUv3
+ *
+ * Return: True if partitioning is possible, false otherwise
+ */
+static bool has_host_pmu_partition_support(void)
+{
+ return has_vhe() &&
+ system_supports_pmuv3();
+}
+
+/**
+ * has_kvm_pmu_partition_support() - If we can enable/disable partition
+ *
+ * Return: true if allowed, false otherwise.
+ */
+bool has_kvm_pmu_partition_support(void)
+{
+ return has_host_pmu_partition_support() &&
+ kvm_supports_guest_pmuv3();
+}
+
+/**
+ * kvm_pmu_partition_enable() - Enable/disable partition flag
+ * @kvm: Pointer to kvm struct
+ * @enable: Whether to enable or disable
+ *
+ * If we want to enable the partition, the guest is free to grab
+ * hardware by accessing PMU registers. Otherwise, the host maintains
+ * control.
+ */
+void kvm_pmu_partition_enable(struct kvm *kvm, bool enable)
+{
+ if (enable)
+ set_bit(KVM_ARCH_FLAG_PARTITION_PMU_ENABLED, &kvm->arch.flags);
+ else
+ clear_bit(KVM_ARCH_FLAG_PARTITION_PMU_ENABLED, &kvm->arch.flags);
+}
+
+/**
+ * kvm_pmu_is_partitioned() - Determine if KVM has a partitioned PMU
+ * @kvm: Pointer to kvm struct
+ *
+ * Return: True if the KVM PMU is partitioned, false otherwise
+ */
+bool kvm_pmu_is_partitioned(struct kvm *kvm)
+{
+ return kvm && test_bit(KVM_ARCH_FLAG_PARTITION_PMU_ENABLED, &kvm->arch.flags);
+}
+
+static u64 kvm_vcpu_pmu_guest_counter_mask(struct kvm_vcpu *vcpu)
+{
+ if (vcpu && kvm_pmu_is_partitioned(vcpu->kvm)) {
+ u64 mask = ARMV8_PMU_CNT_MASK_C;
+ unsigned int guest_counters = vcpu->kvm->arch.nr_pmu_counters;
+
+ if (guest_counters > 0)
+ mask |= GENMASK_ULL(guest_counters - 1, 0);
+
+ return mask;
+ }
+
+ return 0;
+}
+
+/**
+ * kvm_pmu_guest_counter_mask() - Compute bitmask of guest-reserved counters
+ *
+ * Compute the bitmask that selects the guest-reserved counters in the
+ * {PMCNTEN,PMINTEN,PMOVS}{SET,CLR} registers. These are the event counters
+ * in 0..HPMN-1 and the cycle counter (PMCCNTR_EL0).
+ *
+ * Return: Bitmask
+ */
+u64 kvm_pmu_guest_counter_mask(void)
+{
+ return kvm_vcpu_pmu_guest_counter_mask(kvm_get_running_vcpu());
+}
diff --git a/include/kvm/arm_pmu.h b/include/kvm/arm_pmu.h
index 8d51cf40182d6..e06255e46a4d8 100644
--- a/include/kvm/arm_pmu.h
+++ b/include/kvm/arm_pmu.h
@@ -95,6 +95,10 @@ void kvm_vcpu_pmu_resync_el0(void);
#define kvm_vcpu_has_pmuv3_strict(vcpu) \
(vcpu_has_feature(vcpu, KVM_ARM_VCPU_PMU_V3_STRICT))
+bool has_kvm_pmu_partition_support(void);
+void kvm_pmu_partition_enable(struct kvm *kvm, bool enable);
+bool kvm_pmu_is_partitioned(struct kvm *kvm);
+u64 kvm_pmu_guest_counter_mask(void);
/*
* Updates the vcpu's view of the pmu events for this cpu.
* Must be called before every vcpu run after disabling interrupts, to ensure
@@ -129,6 +133,10 @@ static inline u64 kvm_pmu_get_counter_value(struct kvm_vcpu *vcpu,
{
return 0;
}
+static inline bool kvm_pmu_is_partitioned(struct kvm *kvm)
+{
+ return false;
+}
static inline void kvm_pmu_set_counter_value(struct kvm_vcpu *vcpu,
u64 select_idx, u64 val) {}
static inline void kvm_pmu_set_counter_value_user(struct kvm_vcpu *vcpu,
@@ -233,6 +241,18 @@ static inline bool kvm_pmu_counter_is_hyp(struct kvm_vcpu *vcpu, unsigned int id
static inline void kvm_pmu_nested_transition(struct kvm_vcpu *vcpu) {}
+static inline u64 kvm_pmu_guest_counter_mask(void)
+{
+ return 0;
+}
+
+static inline bool has_kvm_pmu_partition_support(void)
+{
+ return false;
+}
+
+static inline void kvm_pmu_partition_enable(struct kvm *kvm, bool enable) {}
+
#endif
#endif
--
2.56.1.708.g77570a6a1d-goog
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v10 12/33] KVM: arm64: Set up FGT for Partitioned PMU
2026-11-19 18:51 [PATCH v10 00/33] ARM64 PMU Partitioning Colton Lewis
` (10 preceding siblings ...)
2026-11-19 18:51 ` [PATCH v10 11/33] KVM: arm64: Add initial scaffolding for Partitioned PMU Colton Lewis
@ 2026-11-19 18:51 ` Colton Lewis
2026-11-19 19:13 ` sashiko-bot
2026-11-19 18:51 ` [PATCH v10 13/33] KVM: arm64: Make __vcpu_rmw_sys_reg() atomic Colton Lewis
` (20 subsequent siblings)
32 siblings, 1 reply; 67+ messages in thread
From: Colton Lewis @ 2026-11-19 18:51 UTC (permalink / raw)
To: Marc Zyngier, Oliver Upton
Cc: Catalin Marinas, Will Deacon, Mark Rutland, Peter Zijlstra,
Ingo Molnar, Paolo Bonzini, Russell King, Jonathan Corbet,
Fuad Tabba, Joey Gouly, Suzuki K Poulose, Zenghui Yu, Shuah Khan,
James Clark, Alexandru Elisei, Mingwei Zhang, Ganapatrao Kulkarni,
kvmarm, linux-arm-kernel, kvm, linux-perf-users, linux-kselftest,
linux-doc, linux-kernel
In order to gain the best performance benefit from partitioning the
PMU, utilize fine grain traps (FEAT_FGT and FEAT_FGT2) to avoid
trapping common PMU register accesses by the guest to remove that
overhead.
Untrapped:
- PMCR_EL0
- PMUSERENR_EL0 (outside nested hypervisor context)
- PMSELR_EL0
- PMCCNTR_EL0
- PMCNTEN_EL0
- PMINTEN_EL1
- PMEVCNTRn_EL0
These are safe to untrap because writing MDCR_EL2.HPMN as this series
will do limits the effect of writes to any of these registers to the
partition of counters 0..HPMN-1. Reads from these registers will not
leak information from between guests as all these registers are
context swapped by a later patch in this series. Reads from these
registers also do not leak any information about the host's hardware
beyond what is promised by PMUv3.
Trapped:
- PMOVS_EL0
- PMEVTYPERn_EL0
- PMCCFILTR_EL0
- PMICNTR_EL0
- PMICFILTR_EL0
- PMCEIDn_EL0
- PMMIR_EL1
- PMUSERENR_EL0 (in nested hypervisor context: writes always, reads when
guest MDCR_EL2.TPM is set)
PMOVS remains trapped so KVM can track overflow IRQs that will need to
be injected into the guest.
PMICNTR and PMICFILTR remain trapped because KVM is not handling them
yet.
PMEVTYPERn remains trapped so KVM can limit which events guests can
count, such as disallowing counting at EL2. PMCCFILTR and PMICFILTR
are special cases of the same.
PMCEIDn and PMMIR remain trapped because they can leak information
specific to the host hardware implementation.
Under nested virtualization, a VHE L1 hypervisor executes both vEL2 and
VHE host EL0 inside is_hyp_ctxt(vcpu) without exiting to L0 across ERET
and SVC transitions. Trap PMUSERENR_EL0 writes via HDFGWTR_EL2 in
is_hyp_ctxt() so L0 can recompute MDCR_EL2 traps whenever L1 enables or
disables EL0 PMU access, and trap PMUSERENR_EL0 reads via HDFGRTR_EL2
when guest MDCR_EL2.TPM is set so MRS PMUSERENR_EL0 from VHE host EL0 is
forwarded to L1.
Signed-off-by: Colton Lewis <coltonlewis@google.com>
---
arch/arm64/kvm/config.c | 58 ++++++++++++++++++++++++++++++++++++++---
1 file changed, 55 insertions(+), 3 deletions(-)
diff --git a/arch/arm64/kvm/config.c b/arch/arm64/kvm/config.c
index adaf26ab93b85..3220ed6fc9453 100644
--- a/arch/arm64/kvm/config.c
+++ b/arch/arm64/kvm/config.c
@@ -1786,12 +1786,64 @@ static void __compute_hfgwtr(struct kvm_vcpu *vcpu)
*vcpu_fgt(vcpu, HFGWTR_EL2) |= HFGWTR_EL2_TCR_EL1;
}
+static void __compute_hdfgrtr(struct kvm_vcpu *vcpu)
+{
+ __compute_fgt(vcpu, HDFGRTR_EL2);
+
+ if (kvm_pmu_is_partitioned(vcpu->kvm)) {
+ u64 set = HDFGRTR_EL2_PMOVS |
+ HDFGRTR_EL2_PMCCFILTR_EL0 |
+ HDFGRTR_EL2_PMEVTYPERn_EL0 |
+ HDFGRTR_EL2_PMCEIDn_EL0 |
+ HDFGRTR_EL2_PMMIR_EL1;
+
+ if (is_hyp_ctxt(vcpu) &&
+ (__vcpu_sys_reg(vcpu, MDCR_EL2) & MDCR_EL2_TPM))
+ set |= HDFGRTR_EL2_PMUSERENR_EL0;
+
+ *vcpu_fgt(vcpu, HDFGRTR_EL2) |= set & ~hdfgrtr_masks.res0;
+ }
+}
+
static void __compute_hdfgwtr(struct kvm_vcpu *vcpu)
{
__compute_fgt(vcpu, HDFGWTR_EL2);
if (is_hyp_ctxt(vcpu))
*vcpu_fgt(vcpu, HDFGWTR_EL2) |= HDFGWTR_EL2_MDSCR_EL1;
+
+ if (kvm_pmu_is_partitioned(vcpu->kvm)) {
+ u64 set = HDFGWTR_EL2_PMOVS |
+ HDFGWTR_EL2_PMCCFILTR_EL0 |
+ HDFGWTR_EL2_PMEVTYPERn_EL0;
+
+ if (is_hyp_ctxt(vcpu))
+ set |= HDFGWTR_EL2_PMUSERENR_EL0;
+
+ *vcpu_fgt(vcpu, HDFGWTR_EL2) |= set & ~hdfgwtr_masks.res0;
+ }
+}
+
+static void __compute_hdfgrtr2(struct kvm_vcpu *vcpu)
+{
+ __compute_fgt(vcpu, HDFGRTR2_EL2);
+
+ if (kvm_pmu_is_partitioned(vcpu->kvm)) {
+ *vcpu_fgt(vcpu, HDFGRTR2_EL2) &=
+ ~(HDFGRTR2_EL2_nPMICFILTR_EL0 |
+ HDFGRTR2_EL2_nPMICNTR_EL0) | hdfgrtr2_masks.res1;
+ }
+}
+
+static void __compute_hdfgwtr2(struct kvm_vcpu *vcpu)
+{
+ __compute_fgt(vcpu, HDFGWTR2_EL2);
+
+ if (kvm_pmu_is_partitioned(vcpu->kvm)) {
+ *vcpu_fgt(vcpu, HDFGWTR2_EL2) &=
+ ~(HDFGWTR2_EL2_nPMICFILTR_EL0 |
+ HDFGWTR2_EL2_nPMICNTR_EL0) | hdfgwtr2_masks.res1;
+ }
}
static void __compute_ich_hfgrtr(struct kvm_vcpu *vcpu)
@@ -1828,7 +1880,7 @@ void kvm_vcpu_load_fgt(struct kvm_vcpu *vcpu)
__compute_fgt(vcpu, HFGRTR_EL2);
__compute_hfgwtr(vcpu);
__compute_fgt(vcpu, HFGITR_EL2);
- __compute_fgt(vcpu, HDFGRTR_EL2);
+ __compute_hdfgrtr(vcpu);
__compute_hdfgwtr(vcpu);
__compute_fgt(vcpu, HAFGRTR_EL2);
@@ -1836,8 +1888,8 @@ void kvm_vcpu_load_fgt(struct kvm_vcpu *vcpu)
__compute_fgt(vcpu, HFGRTR2_EL2);
__compute_fgt(vcpu, HFGWTR2_EL2);
__compute_fgt(vcpu, HFGITR2_EL2);
- __compute_fgt(vcpu, HDFGRTR2_EL2);
- __compute_fgt(vcpu, HDFGWTR2_EL2);
+ __compute_hdfgrtr2(vcpu);
+ __compute_hdfgwtr2(vcpu);
}
if (cpus_have_final_cap(ARM64_HAS_GICV5_CPUIF)) {
--
2.56.1.708.g77570a6a1d-goog
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v10 13/33] KVM: arm64: Make __vcpu_rmw_sys_reg() atomic
2026-11-19 18:51 [PATCH v10 00/33] ARM64 PMU Partitioning Colton Lewis
` (11 preceding siblings ...)
2026-11-19 18:51 ` [PATCH v10 12/33] KVM: arm64: Set up FGT " Colton Lewis
@ 2026-11-19 18:51 ` Colton Lewis
2026-11-19 18:58 ` sashiko-bot
2026-11-19 18:51 ` [PATCH v10 14/33] KVM: arm64: Add Partitioned PMU register trap handlers Colton Lewis
` (19 subsequent siblings)
32 siblings, 1 reply; 67+ messages in thread
From: Colton Lewis @ 2026-11-19 18:51 UTC (permalink / raw)
To: Marc Zyngier, Oliver Upton
Cc: Catalin Marinas, Will Deacon, Mark Rutland, Peter Zijlstra,
Ingo Molnar, Paolo Bonzini, Russell King, Jonathan Corbet,
Fuad Tabba, Joey Gouly, Suzuki K Poulose, Zenghui Yu, Shuah Khan,
James Clark, Alexandru Elisei, Mingwei Zhang, Ganapatrao Kulkarni,
kvmarm, linux-arm-kernel, kvm, linux-perf-users, linux-kselftest,
linux-doc, linux-kernel
Use try_cmpxchg_local() in __vcpu_rmw_sys_reg() so read-modify-write
updates to vCPU system registers are atomic with respect to local
interrupts and GICv3 pseudo-NMIs. This prevents lost updates when a PMU
overflow handler (which may run as a pseudo-NMI that is not masked by
local_irq_save()) updates PMOVSSET_EL0 concurrently with vCPU register
accesses.
Signed-off-by: Colton Lewis <coltonlewis@google.com>
---
arch/arm64/include/asm/kvm_host.h | 14 ++++++++------
1 file changed, 8 insertions(+), 6 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 89683a690828e..103c25001c0d2 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -1313,12 +1313,14 @@ u64 kvm_vcpu_apply_reg_masks(const struct kvm_vcpu *, enum vcpu_sysreg, u64);
#define __vcpu_rmw_sys_reg(v, r, op, val) \
do { \
const struct kvm_cpu_context *ctxt = &(v)->arch.ctxt; \
- u64 __v = ctxt_sys_reg(ctxt, (r)); \
- __v op (val); \
- if (vcpu_has_nv((v)) && (r) >= __SANITISED_REG_START__) \
- __v = kvm_vcpu_apply_reg_masks((v), (r), __v); \
- \
- ctxt_sys_reg(ctxt, (r)) = __v; \
+ u64 __old = READ_ONCE(ctxt_sys_reg(ctxt, (r))); \
+ u64 __new; \
+ do { \
+ __new = __old; \
+ __new op (val); \
+ if (vcpu_has_nv((v)) && (r) >= __SANITISED_REG_START__) \
+ __new = kvm_vcpu_apply_reg_masks((v), (r), __new); \
+ } while (!try_cmpxchg_local(__ctxt_sys_reg(ctxt, (r)), &__old, __new)); \
} while (0)
#define __vcpu_sys_reg(v,r) \
--
2.56.1.708.g77570a6a1d-goog
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v10 14/33] KVM: arm64: Add Partitioned PMU register trap handlers
2026-11-19 18:51 [PATCH v10 00/33] ARM64 PMU Partitioning Colton Lewis
` (12 preceding siblings ...)
2026-11-19 18:51 ` [PATCH v10 13/33] KVM: arm64: Make __vcpu_rmw_sys_reg() atomic Colton Lewis
@ 2026-11-19 18:51 ` Colton Lewis
2026-11-19 19:21 ` sashiko-bot
2026-11-19 18:51 ` [PATCH v10 15/33] KVM: arm64: Construct mdcr_el2 in a local variable in kvm_arm_setup_mdcr_el2() Colton Lewis
` (18 subsequent siblings)
32 siblings, 1 reply; 67+ messages in thread
From: Colton Lewis @ 2026-11-19 18:51 UTC (permalink / raw)
To: Marc Zyngier, Oliver Upton
Cc: Catalin Marinas, Will Deacon, Mark Rutland, Peter Zijlstra,
Ingo Molnar, Paolo Bonzini, Russell King, Jonathan Corbet,
Fuad Tabba, Joey Gouly, Suzuki K Poulose, Zenghui Yu, Shuah Khan,
James Clark, Alexandru Elisei, Mingwei Zhang, Ganapatrao Kulkarni,
kvmarm, linux-arm-kernel, kvm, linux-perf-users, linux-kselftest,
linux-doc, linux-kernel
Even when a VM uses a Partitioned PMU, PMU register accesses still trap
to EL2 when the host lacks FEAT_FGT, when accessing registers that
remain trapped via FGT (PMOVS, PMEVTYPER<n>_EL0, PMCCFILTR_EL0), or when
MDCR_EL2.TPM / MDCR_EL2.TPMCR is active for a nested context.
Add kvm_pmu_direct_reg_read() and kvm_pmu_direct_reg_write() in
arch/arm64/kvm/pmu-direct.c under a single guard(preempt)() per call and
dispatch partitioned accesses from pmu_reg_read() and pmu_reg_write() in
arch/arm64/kvm/sys_regs.c:
- Read and write live hardware registers when the partitioned PMU is
loaded on the local CPU (masking counter bitmaps to the guest
partition), and fall back to the shadow sys_regs[] array otherwise.
- Clear guest overflow bits in both physical pmovsclr_el0 and shadow
PMOVSSET_EL0 on PMOVSCLR_EL0 writes, and harvest live hardware guest
overflow bits into shadow PMOVSSET_EL0 on PMOVS reads.
- Record filtered guest PMEVTYPER<n>_EL0 and PMCCFILTR_EL0 values in
sys_regs[]; a subsequent patch enforces the KVM PMU event filter in
hardware.
- Add kvm_pmu_get_pmselr() in arch/arm64/kvm/emulate-nested.c so
check_mdcr_hpmn() evaluates the live PMSELR_EL0 counter selector on
trapped PMXEVTYPER_EL0 and PMXEVCNTR_EL0 accesses.
Signed-off-by: Colton Lewis <coltonlewis@google.com>
---
arch/arm64/kvm/emulate-nested.c | 12 +-
arch/arm64/kvm/pmu-direct.c | 252 ++++++++++++++++++++++++++++++++
arch/arm64/kvm/sys_regs.c | 157 ++++++++++++++------
include/kvm/arm_pmu.h | 13 ++
4 files changed, 383 insertions(+), 51 deletions(-)
diff --git a/arch/arm64/kvm/emulate-nested.c b/arch/arm64/kvm/emulate-nested.c
index 2bec816683614..1cd2f42452fe7 100644
--- a/arch/arm64/kvm/emulate-nested.c
+++ b/arch/arm64/kvm/emulate-nested.c
@@ -562,12 +562,19 @@ static enum trap_behaviour check_cptr_tta(struct kvm_vcpu *vcpu)
return BEHAVE_HANDLE_LOCALLY;
}
+static u64 kvm_pmu_get_pmselr(struct kvm_vcpu *vcpu)
+{
+ if (kvm_pmu_is_partitioned(vcpu->kvm))
+ return kvm_pmu_direct_reg_read(vcpu, PMSELR_EL0);
+
+ return __vcpu_sys_reg(vcpu, PMSELR_EL0);
+}
+
static enum trap_behaviour check_mdcr_hpmn(struct kvm_vcpu *vcpu)
{
u32 sysreg = esr_sys64_to_sysreg(kvm_vcpu_get_esr(vcpu));
unsigned int idx;
-
switch (sysreg) {
case SYS_PMEVTYPERn_EL0(0) ... SYS_PMEVTYPERn_EL0(30):
case SYS_PMEVCNTRn_EL0(0) ... SYS_PMEVCNTRn_EL0(30):
@@ -575,8 +582,7 @@ static enum trap_behaviour check_mdcr_hpmn(struct kvm_vcpu *vcpu)
break;
case SYS_PMXEVTYPER_EL0:
case SYS_PMXEVCNTR_EL0:
- idx = SYS_FIELD_GET(PMSELR_EL0, SEL,
- __vcpu_sys_reg(vcpu, PMSELR_EL0));
+ idx = SYS_FIELD_GET(PMSELR_EL0, SEL, kvm_pmu_get_pmselr(vcpu));
break;
default:
/* Someone used this trap helper for something else... */
diff --git a/arch/arm64/kvm/pmu-direct.c b/arch/arm64/kvm/pmu-direct.c
index b1c180ae79d9d..13f5bcc2a9084 100644
--- a/arch/arm64/kvm/pmu-direct.c
+++ b/arch/arm64/kvm/pmu-direct.c
@@ -4,7 +4,9 @@
* Author: Colton Lewis <coltonlewis@google.com>
*/
+#include <linux/cleanup.h>
#include <linux/kvm_host.h>
+#include <linux/lockdep.h>
#include <linux/perf/arm_pmu.h>
#include <linux/perf/arm_pmuv3.h>
@@ -64,6 +66,17 @@ bool kvm_pmu_is_partitioned(struct kvm *kvm)
return kvm && test_bit(KVM_ARCH_FLAG_PARTITION_PMU_ENABLED, &kvm->arch.flags);
}
+static bool kvm_pmu_part_on_cpu(struct kvm_vcpu *vcpu)
+{
+ struct arm_pmu *pmu = vcpu->kvm->arch.arm_pmu;
+
+ lockdep_assert_preemption_disabled();
+
+ return kvm_pmu_is_partitioned(vcpu->kvm) &&
+ vcpu == kvm_get_running_vcpu() && pmu &&
+ cpumask_test_cpu(smp_processor_id(), &pmu->supported_cpus);
+}
+
static u64 kvm_vcpu_pmu_guest_counter_mask(struct kvm_vcpu *vcpu)
{
if (vcpu && kvm_pmu_is_partitioned(vcpu->kvm)) {
@@ -79,6 +92,245 @@ static u64 kvm_vcpu_pmu_guest_counter_mask(struct kvm_vcpu *vcpu)
return 0;
}
+static void kvm_pmu_direct_pmcr_write(struct kvm_vcpu *vcpu, u64 val)
+{
+ bool reset_p = val & ARMV8_PMU_PMCR_P;
+ bool reset_c = val & ARMV8_PMU_PMCR_C;
+ unsigned long mask;
+ u64 pmcr_mask;
+ int i;
+
+ lockdep_assert_preemption_disabled();
+
+ if (!kvm_has_feat(vcpu->kvm, ID_AA64DFR0_EL1, PMUVer, V3P5))
+ val &= ~ARMV8_PMU_PMCR_LP;
+
+ pmcr_mask = ARMV8_PMU_PMCR_MASK &
+ ~(ARMV8_PMU_PMCR_P | ARMV8_PMU_PMCR_C);
+ __vcpu_rmw_sys_reg(vcpu, PMCR_EL0, &=, ~ARMV8_PMU_PMCR_MASK);
+ __vcpu_rmw_sys_reg(vcpu, PMCR_EL0, |=, val & pmcr_mask);
+
+ if (reset_c)
+ __vcpu_assign_sys_reg(vcpu, PMCCNTR_EL0, 0);
+
+ mask = kvm_pmu_accessible_counter_mask(vcpu) &
+ ~BIT(ARMV8_PMU_CYCLE_IDX);
+ if (reset_p) {
+ for_each_set_bit(i, &mask, ARMV8_PMU_MAX_GENERAL_COUNTERS)
+ __vcpu_assign_sys_reg(vcpu, PMEVCNTR0_EL0 + i, 0);
+ }
+
+ if (!kvm_pmu_part_on_cpu(vcpu))
+ return;
+
+ if (reset_c)
+ write_pmccntr(0);
+
+ if (reset_p) {
+ for_each_set_bit(i, &mask, ARMV8_PMU_MAX_GENERAL_COUNTERS)
+ write_pmevcntrn(i, 0);
+ }
+
+ write_sysreg(val & ~(ARMV8_PMU_PMCR_P | ARMV8_PMU_PMCR_C), pmcr_el0);
+}
+
+static u64 kvm_pmu_direct_pmcr_read(struct kvm_vcpu *vcpu)
+{
+ u64 n = vcpu->kvm->arch.nr_pmu_counters;
+ u64 pmcr;
+
+ lockdep_assert_preemption_disabled();
+
+ if (!kvm_pmu_part_on_cpu(vcpu))
+ return kvm_vcpu_read_pmcr(vcpu);
+
+ pmcr = read_sysreg(pmcr_el0);
+ if (!kvm_has_feat(vcpu->kvm, ID_AA64DFR0_EL1, PMUVer, V3P5))
+ pmcr &= ~ARMV8_PMU_PMCR_LP;
+
+ if (vcpu_has_nv(vcpu) && !vcpu_is_el2(vcpu))
+ n = FIELD_GET(MDCR_EL2_HPMN, __vcpu_sys_reg(vcpu, MDCR_EL2));
+
+ return u64_replace_bits(pmcr, n, ARMV8_PMU_PMCR_N);
+}
+
+static u64 kvm_pmu_direct_cntr_read(struct kvm_vcpu *vcpu, u8 idx)
+{
+ lockdep_assert_preemption_disabled();
+
+ if (!kvm_pmu_part_on_cpu(vcpu))
+ return __vcpu_sys_reg(vcpu, PMEVCNTR0_EL0 + idx);
+
+ if (idx == ARMV8_PMU_CYCLE_IDX)
+ return read_pmccntr();
+
+ return read_pmevcntrn(idx);
+}
+
+static void kvm_pmu_direct_cntr_write(struct kvm_vcpu *vcpu, u8 idx, u64 val)
+{
+ lockdep_assert_preemption_disabled();
+
+ __vcpu_assign_sys_reg(vcpu, PMEVCNTR0_EL0 + idx, val);
+ if (!kvm_pmu_part_on_cpu(vcpu))
+ return;
+
+ if (idx == ARMV8_PMU_CYCLE_IDX)
+ write_pmccntr(val);
+ else
+ write_pmevcntrn(idx, val);
+}
+
+/**
+ * kvm_pmu_direct_reg_read() - Read a guest PMU register for Partitioned PMU
+ * @vcpu: Pointer to vcpu struct
+ * @reg: vCPU system register index
+ *
+ * Return: Register value read from hardware or shadow state
+ */
+u64 kvm_pmu_direct_reg_read(struct kvm_vcpu *vcpu, unsigned int reg)
+{
+ unsigned long flags;
+ u64 val = 0;
+
+ guard(preempt)();
+
+ switch (reg) {
+ case PMCR_EL0:
+ return kvm_pmu_direct_pmcr_read(vcpu);
+ case PMSELR_EL0:
+ if (kvm_pmu_part_on_cpu(vcpu) &&
+ !(vcpu->arch.mdcr_el2 & MDCR_EL2_TPM))
+ return read_sysreg(pmselr_el0);
+ return __vcpu_sys_reg(vcpu, reg);
+ case PMEVCNTR0_EL0 ... PMCCNTR_EL0:
+ return kvm_pmu_direct_cntr_read(vcpu, reg - PMEVCNTR0_EL0);
+ case PMEVTYPER0_EL0 ... PMCCFILTR_EL0:
+ return __vcpu_sys_reg(vcpu, reg);
+ case PMCNTENSET_EL0:
+ if (kvm_pmu_part_on_cpu(vcpu))
+ return read_sysreg(pmcntenset_el0) &
+ kvm_vcpu_pmu_guest_counter_mask(vcpu);
+ return __vcpu_sys_reg(vcpu, reg);
+ case PMINTENSET_EL1:
+ if (kvm_pmu_part_on_cpu(vcpu))
+ return read_sysreg(pmintenset_el1) &
+ kvm_vcpu_pmu_guest_counter_mask(vcpu);
+ return __vcpu_sys_reg(vcpu, reg);
+ case PMOVSSET_EL0:
+ local_irq_save(flags);
+ if (kvm_pmu_part_on_cpu(vcpu)) {
+ u64 hw_ovf = read_sysreg(pmovsset_el0) &
+ kvm_vcpu_pmu_guest_counter_mask(vcpu);
+
+ if (hw_ovf) {
+ __vcpu_rmw_sys_reg(vcpu, PMOVSSET_EL0, |=, hw_ovf);
+ write_sysreg(hw_ovf, pmovsclr_el0);
+ }
+ }
+ val = __vcpu_sys_reg(vcpu, reg);
+ local_irq_restore(flags);
+ return val;
+ case PMUSERENR_EL0:
+ if (kvm_pmu_part_on_cpu(vcpu) &&
+ !(vcpu->arch.mdcr_el2 & MDCR_EL2_TPM))
+ return read_sysreg(pmuserenr_el0);
+ return __vcpu_sys_reg(vcpu, reg);
+ default:
+ WARN_ON(1);
+ return 0;
+ }
+}
+
+/**
+ * kvm_pmu_direct_reg_write() - Write a guest PMU register for Partitioned PMU
+ * @vcpu: Pointer to vcpu struct
+ * @reg: vCPU system register index
+ * @val: Value to write
+ * @set: True when setting bits, false when clearing bits in mask registers
+ */
+void kvm_pmu_direct_reg_write(struct kvm_vcpu *vcpu, unsigned int reg,
+ u64 val, bool set)
+{
+ unsigned long flags;
+ u64 mask;
+ int idx;
+
+ guard(preempt)();
+
+ switch (reg) {
+ case PMCR_EL0:
+ kvm_pmu_direct_pmcr_write(vcpu, val);
+ break;
+ case PMSELR_EL0:
+ if (kvm_pmu_part_on_cpu(vcpu) &&
+ !(vcpu->arch.mdcr_el2 & MDCR_EL2_TPM))
+ write_sysreg(val, pmselr_el0);
+ __vcpu_assign_sys_reg(vcpu, reg, val);
+ break;
+ case PMEVCNTR0_EL0 ... PMCCNTR_EL0:
+ idx = reg - PMEVCNTR0_EL0;
+ if (vcpu_mode_is_32bit(vcpu) && idx != ARMV8_PMU_CYCLE_IDX)
+ val = (kvm_pmu_direct_cntr_read(vcpu, idx) &
+ GENMASK_ULL(63, 32)) |
+ lower_32_bits(val);
+ kvm_pmu_direct_cntr_write(vcpu, idx, val);
+ break;
+ case PMEVTYPER0_EL0 ... PMCCFILTR_EL0:
+ mask = kvm_pmu_evtyper_mask(vcpu->kvm);
+ if (reg == PMCCFILTR_EL0)
+ mask &= ~kvm_pmu_event_mask(vcpu->kvm);
+ __vcpu_assign_sys_reg(vcpu, reg, val & mask);
+ break;
+ case PMCNTENSET_EL0:
+ val &= kvm_vcpu_pmu_guest_counter_mask(vcpu);
+ if (set) {
+ __vcpu_rmw_sys_reg(vcpu, PMCNTENSET_EL0, |=, val);
+ if (kvm_pmu_part_on_cpu(vcpu))
+ write_sysreg(val, pmcntenset_el0);
+ } else {
+ __vcpu_rmw_sys_reg(vcpu, PMCNTENSET_EL0, &=, ~val);
+ if (kvm_pmu_part_on_cpu(vcpu))
+ write_sysreg(val, pmcntenclr_el0);
+ }
+ break;
+ case PMINTENSET_EL1:
+ val &= kvm_vcpu_pmu_guest_counter_mask(vcpu);
+ if (set) {
+ __vcpu_rmw_sys_reg(vcpu, PMINTENSET_EL1, |=, val);
+ if (kvm_pmu_part_on_cpu(vcpu))
+ write_sysreg(val, pmintenset_el1);
+ } else {
+ __vcpu_rmw_sys_reg(vcpu, PMINTENSET_EL1, &=, ~val);
+ if (kvm_pmu_part_on_cpu(vcpu))
+ write_sysreg(val, pmintenclr_el1);
+ }
+ break;
+ case PMOVSSET_EL0:
+ local_irq_save(flags);
+ if (set) {
+ __vcpu_rmw_sys_reg(vcpu, PMOVSSET_EL0, |=, val);
+ } else {
+ mask = kvm_vcpu_pmu_guest_counter_mask(vcpu);
+ if (kvm_pmu_part_on_cpu(vcpu))
+ write_sysreg(val & mask, pmovsclr_el0);
+ __vcpu_rmw_sys_reg(vcpu, PMOVSSET_EL0, &=, ~val);
+ }
+ local_irq_restore(flags);
+ break;
+ case PMUSERENR_EL0:
+ val &= ARMV8_PMU_USERENR_MASK;
+ if (kvm_pmu_part_on_cpu(vcpu) &&
+ !(vcpu->arch.mdcr_el2 & MDCR_EL2_TPM))
+ write_sysreg(val, pmuserenr_el0);
+ __vcpu_assign_sys_reg(vcpu, reg, val);
+ break;
+ default:
+ WARN_ON(1);
+ break;
+ }
+}
+
/**
* kvm_pmu_guest_counter_mask() - Compute bitmask of guest-reserved counters
*
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 2d9ae781acff0..db6f3adcaae7c 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1089,9 +1089,85 @@ static u64 reset_pmcr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r)
return __vcpu_sys_reg(vcpu, r->reg);
}
+static u64 pmu_reg_read(struct kvm_vcpu *vcpu, enum vcpu_sysreg reg)
+{
+ if (kvm_pmu_is_partitioned(vcpu->kvm))
+ return kvm_pmu_direct_reg_read(vcpu, reg);
+
+ switch (reg) {
+ case PMCR_EL0:
+ return kvm_vcpu_read_pmcr(vcpu);
+ case PMEVCNTR0_EL0 ... PMCCNTR_EL0:
+ return kvm_pmu_get_counter_value(vcpu, reg - PMEVCNTR0_EL0);
+ case PMSELR_EL0:
+ case PMEVTYPER0_EL0 ... PMCCFILTR_EL0:
+ case PMCNTENSET_EL0:
+ case PMINTENSET_EL1:
+ case PMOVSSET_EL0:
+ case PMUSERENR_EL0:
+ return __vcpu_sys_reg(vcpu, reg);
+ default:
+ WARN_ON(1);
+ return 0;
+ }
+}
+
+static void pmu_reg_write(struct kvm_vcpu *vcpu, enum vcpu_sysreg reg, u64 val,
+ bool set)
+{
+ int idx;
+
+ if (kvm_pmu_is_partitioned(vcpu->kvm)) {
+ kvm_pmu_direct_reg_write(vcpu, reg, val, set);
+ return;
+ }
+
+ switch (reg) {
+ case PMCR_EL0:
+ kvm_pmu_handle_pmcr(vcpu, val);
+ break;
+ case PMSELR_EL0:
+ __vcpu_assign_sys_reg(vcpu, reg, val);
+ break;
+ case PMEVCNTR0_EL0 ... PMCCNTR_EL0:
+ kvm_pmu_set_counter_value(vcpu, reg - PMEVCNTR0_EL0, val);
+ break;
+ case PMEVTYPER0_EL0 ... PMCCFILTR_EL0:
+ idx = reg - PMEVTYPER0_EL0;
+ kvm_pmu_set_counter_event_type(vcpu, val, idx);
+ kvm_vcpu_pmu_restore_guest(vcpu);
+ break;
+ case PMCNTENSET_EL0:
+ if (set)
+ __vcpu_rmw_sys_reg(vcpu, PMCNTENSET_EL0, |=, val);
+ else
+ __vcpu_rmw_sys_reg(vcpu, PMCNTENSET_EL0, &=, ~val);
+ kvm_pmu_reprogram_counter_mask(vcpu, val);
+ break;
+ case PMINTENSET_EL1:
+ if (set)
+ __vcpu_rmw_sys_reg(vcpu, PMINTENSET_EL1, |=, val);
+ else
+ __vcpu_rmw_sys_reg(vcpu, PMINTENSET_EL1, &=, ~val);
+ break;
+ case PMOVSSET_EL0:
+ if (set)
+ __vcpu_rmw_sys_reg(vcpu, PMOVSSET_EL0, |=, val);
+ else
+ __vcpu_rmw_sys_reg(vcpu, PMOVSSET_EL0, &=, ~val);
+ break;
+ case PMUSERENR_EL0:
+ __vcpu_assign_sys_reg(vcpu, reg, val & ARMV8_PMU_USERENR_MASK);
+ break;
+ default:
+ WARN_ON(1);
+ break;
+ }
+}
+
static bool check_pmu_access_disabled(struct kvm_vcpu *vcpu, u64 flags)
{
- u64 reg = __vcpu_sys_reg(vcpu, PMUSERENR_EL0);
+ u64 reg = pmu_reg_read(vcpu, PMUSERENR_EL0);
bool enabled = (reg & flags) || vcpu_mode_priv(vcpu);
if (!enabled)
@@ -1130,18 +1206,17 @@ static bool access_pmcr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
if (p->is_write) {
/*
- * Only update writeable bits of PMCR (continuing into
- * kvm_pmu_handle_pmcr() as well)
+ * Only update writeable bits of PMCR
*/
- val = kvm_vcpu_read_pmcr(vcpu);
+ val = pmu_reg_read(vcpu, PMCR_EL0);
val &= ~ARMV8_PMU_PMCR_MASK;
val |= p->regval & ARMV8_PMU_PMCR_MASK;
if (!kvm_supports_32bit_el0())
val |= ARMV8_PMU_PMCR_LC;
- kvm_pmu_handle_pmcr(vcpu, val);
+ pmu_reg_write(vcpu, PMCR_EL0, val, 0);
} else {
/* PMCR.P & PMCR.C are RAZ */
- val = kvm_vcpu_read_pmcr(vcpu)
+ val = pmu_reg_read(vcpu, PMCR_EL0)
& ~(ARMV8_PMU_PMCR_P | ARMV8_PMU_PMCR_C);
p->regval = val;
}
@@ -1156,10 +1231,10 @@ static bool access_pmselr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
return false;
if (p->is_write)
- __vcpu_assign_sys_reg(vcpu, PMSELR_EL0, p->regval);
+ pmu_reg_write(vcpu, PMSELR_EL0, p->regval, 0);
else
/* return PMSELR.SEL field */
- p->regval = __vcpu_sys_reg(vcpu, PMSELR_EL0)
+ p->regval = pmu_reg_read(vcpu, PMSELR_EL0)
& PMSELR_EL0_SEL_MASK;
return true;
@@ -1236,6 +1311,7 @@ static bool access_pmu_evcntr(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
+ enum vcpu_sysreg reg;
u64 idx = ~0UL;
if (r->CRn == 9 && r->CRm == 13) {
@@ -1245,7 +1321,7 @@ static bool access_pmu_evcntr(struct kvm_vcpu *vcpu,
return false;
idx = SYS_FIELD_GET(PMSELR_EL0, SEL,
- __vcpu_sys_reg(vcpu, PMSELR_EL0));
+ pmu_reg_read(vcpu, PMSELR_EL0));
} else if (r->Op2 == 0) {
/* PMCCNTR_EL0 */
if (pmu_access_cycle_counter_el0_disabled(vcpu))
@@ -1273,13 +1349,15 @@ static bool access_pmu_evcntr(struct kvm_vcpu *vcpu,
if (!pmu_counter_idx_valid(vcpu, idx))
return false;
+ reg = PMEVCNTR0_EL0 + idx;
+
if (p->is_write) {
if (pmu_access_el0_disabled(vcpu))
return false;
- kvm_pmu_set_counter_value(vcpu, idx, p->regval);
+ pmu_reg_write(vcpu, reg, p->regval, 0);
} else {
- p->regval = kvm_pmu_get_counter_value(vcpu, idx);
+ p->regval = pmu_reg_read(vcpu, reg);
}
return true;
@@ -1295,7 +1373,7 @@ static bool access_pmu_evtyper(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
if (r->CRn == 9 && r->CRm == 13 && r->Op2 == 1) {
/* PMXEVTYPER_EL0 */
- idx = SYS_FIELD_GET(PMSELR_EL0, SEL, __vcpu_sys_reg(vcpu, PMSELR_EL0));
+ idx = SYS_FIELD_GET(PMSELR_EL0, SEL, pmu_reg_read(vcpu, PMSELR_EL0));
reg = PMEVTYPER0_EL0 + idx;
} else if (r->CRn == 14 && (r->CRm & 12) == 12) {
idx = ((r->CRm & 3) << 3) | (r->Op2 & 7);
@@ -1311,12 +1389,10 @@ static bool access_pmu_evtyper(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
if (!pmu_counter_idx_valid(vcpu, idx))
return false;
- if (p->is_write) {
- kvm_pmu_set_counter_event_type(vcpu, p->regval, idx);
- kvm_vcpu_pmu_restore_guest(vcpu);
- } else {
- p->regval = __vcpu_sys_reg(vcpu, reg);
- }
+ if (p->is_write)
+ pmu_reg_write(vcpu, reg, p->regval, 0);
+ else
+ p->regval = pmu_reg_read(vcpu, reg);
return true;
}
@@ -1350,16 +1426,9 @@ static bool access_pmcnten(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
mask = kvm_pmu_accessible_counter_mask(vcpu);
if (p->is_write) {
val = p->regval & mask;
- if (r->Op2 & 0x1)
- /* accessing PMCNTENSET_EL0 */
- __vcpu_rmw_sys_reg(vcpu, PMCNTENSET_EL0, |=, val);
- else
- /* accessing PMCNTENCLR_EL0 */
- __vcpu_rmw_sys_reg(vcpu, PMCNTENSET_EL0, &=, ~val);
-
- kvm_pmu_reprogram_counter_mask(vcpu, val);
+ pmu_reg_write(vcpu, PMCNTENSET_EL0, val, r->Op2 & 0x1);
} else {
- p->regval = __vcpu_sys_reg(vcpu, PMCNTENSET_EL0) & mask;
+ p->regval = pmu_reg_read(vcpu, PMCNTENSET_EL0) & mask;
}
return true;
@@ -1368,22 +1437,17 @@ static bool access_pmcnten(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
static bool access_pminten(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
- u64 mask = kvm_pmu_accessible_counter_mask(vcpu);
+ u64 val, mask;
if (check_pmu_access_disabled(vcpu, 0))
return false;
+ mask = kvm_pmu_accessible_counter_mask(vcpu);
if (p->is_write) {
- u64 val = p->regval & mask;
-
- if (r->Op2 & 0x1)
- /* accessing PMINTENSET_EL1 */
- __vcpu_rmw_sys_reg(vcpu, PMINTENSET_EL1, |=, val);
- else
- /* accessing PMINTENCLR_EL1 */
- __vcpu_rmw_sys_reg(vcpu, PMINTENSET_EL1, &=, ~val);
+ val = p->regval & mask;
+ pmu_reg_write(vcpu, PMINTENSET_EL1, val, r->Op2 & 0x1);
} else {
- p->regval = __vcpu_sys_reg(vcpu, PMINTENSET_EL1) & mask;
+ p->regval = pmu_reg_read(vcpu, PMINTENSET_EL1) & mask;
}
return true;
@@ -1450,20 +1514,18 @@ static int set_pmmir(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r,
static bool access_pmovs(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
- u64 mask = kvm_pmu_accessible_counter_mask(vcpu);
+ u64 val, mask;
if (pmu_access_el0_disabled(vcpu))
return false;
+ mask = kvm_pmu_accessible_counter_mask(vcpu);
+
if (p->is_write) {
- if (r->CRm & 0x2)
- /* accessing PMOVSSET_EL0 */
- __vcpu_rmw_sys_reg(vcpu, PMOVSSET_EL0, |=, (p->regval & mask));
- else
- /* accessing PMOVSCLR_EL0 */
- __vcpu_rmw_sys_reg(vcpu, PMOVSSET_EL0, &=, ~(p->regval & mask));
+ val = p->regval & mask;
+ pmu_reg_write(vcpu, PMOVSSET_EL0, val, r->CRm & 0x2);
} else {
- p->regval = __vcpu_sys_reg(vcpu, PMOVSSET_EL0) & mask;
+ p->regval = pmu_reg_read(vcpu, PMOVSSET_EL0) & mask;
}
return true;
@@ -1492,10 +1554,9 @@ static bool access_pmuserenr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
if (!vcpu_mode_priv(vcpu))
return undef_access(vcpu, p, r);
- __vcpu_assign_sys_reg(vcpu, PMUSERENR_EL0,
- (p->regval & ARMV8_PMU_USERENR_MASK));
+ pmu_reg_write(vcpu, PMUSERENR_EL0, p->regval & ARMV8_PMU_USERENR_MASK, 0);
} else {
- p->regval = __vcpu_sys_reg(vcpu, PMUSERENR_EL0)
+ p->regval = pmu_reg_read(vcpu, PMUSERENR_EL0)
& ARMV8_PMU_USERENR_MASK;
}
diff --git a/include/kvm/arm_pmu.h b/include/kvm/arm_pmu.h
index e06255e46a4d8..276ab74fa0208 100644
--- a/include/kvm/arm_pmu.h
+++ b/include/kvm/arm_pmu.h
@@ -98,6 +98,9 @@ void kvm_vcpu_pmu_resync_el0(void);
bool has_kvm_pmu_partition_support(void);
void kvm_pmu_partition_enable(struct kvm *kvm, bool enable);
bool kvm_pmu_is_partitioned(struct kvm *kvm);
+u64 kvm_pmu_direct_reg_read(struct kvm_vcpu *vcpu, unsigned int reg);
+void kvm_pmu_direct_reg_write(struct kvm_vcpu *vcpu, unsigned int reg,
+ u64 val, bool set);
u64 kvm_pmu_guest_counter_mask(void);
/*
* Updates the vcpu's view of the pmu events for this cpu.
@@ -137,6 +140,16 @@ static inline bool kvm_pmu_is_partitioned(struct kvm *kvm)
{
return false;
}
+
+static inline u64 kvm_pmu_direct_reg_read(struct kvm_vcpu *vcpu,
+ unsigned int reg)
+{
+ return 0;
+}
+
+static inline void kvm_pmu_direct_reg_write(struct kvm_vcpu *vcpu,
+ unsigned int reg, u64 val,
+ bool set) {}
static inline void kvm_pmu_set_counter_value(struct kvm_vcpu *vcpu,
u64 select_idx, u64 val) {}
static inline void kvm_pmu_set_counter_value_user(struct kvm_vcpu *vcpu,
--
2.56.1.708.g77570a6a1d-goog
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v10 15/33] KVM: arm64: Construct mdcr_el2 in a local variable in kvm_arm_setup_mdcr_el2()
2026-11-19 18:51 [PATCH v10 00/33] ARM64 PMU Partitioning Colton Lewis
` (13 preceding siblings ...)
2026-11-19 18:51 ` [PATCH v10 14/33] KVM: arm64: Add Partitioned PMU register trap handlers Colton Lewis
@ 2026-11-19 18:51 ` Colton Lewis
2026-11-19 18:56 ` sashiko-bot
2026-11-19 18:51 ` [PATCH v10 16/33] KVM: arm64: Set up MDCR_EL2 to handle a Partitioned PMU Colton Lewis
` (17 subsequent siblings)
32 siblings, 1 reply; 67+ messages in thread
From: Colton Lewis @ 2026-11-19 18:51 UTC (permalink / raw)
To: Marc Zyngier, Oliver Upton
Cc: Catalin Marinas, Will Deacon, Mark Rutland, Peter Zijlstra,
Ingo Molnar, Paolo Bonzini, Russell King, Jonathan Corbet,
Fuad Tabba, Joey Gouly, Suzuki K Poulose, Zenghui Yu, Shuah Khan,
James Clark, Alexandru Elisei, Mingwei Zhang, Ganapatrao Kulkarni,
kvmarm, linux-arm-kernel, kvm, linux-perf-users, linux-kselftest,
linux-doc, linux-kernel
Construct the new MDCR_EL2 value in a local variable before writing it
once to vcpu->arch.mdcr_el2. This avoids exposing intermediate states in
vcpu->arch.mdcr_el2 to interrupt or NMI handlers that may inspect it,
and simplifies upcoming Partitioned PMU modifications to MDCR_EL2.
Signed-off-by: Colton Lewis <coltonlewis@google.com>
---
arch/arm64/include/asm/kvm_nested.h | 2 +-
arch/arm64/kvm/debug.c | 27 +++++++++++++++------------
arch/arm64/kvm/nested.c | 6 +++---
3 files changed, 19 insertions(+), 16 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
index 0f899addafa8b..a1ad046428a2d 100644
--- a/arch/arm64/include/asm/kvm_nested.h
+++ b/arch/arm64/include/asm/kvm_nested.h
@@ -90,7 +90,7 @@ extern void check_nested_vcpu_requests(struct kvm_vcpu *vcpu);
extern void kvm_nested_flush_hwstate(struct kvm_vcpu *vcpu);
extern void kvm_nested_sync_hwstate(struct kvm_vcpu *vcpu);
-extern void kvm_nested_setup_mdcr_el2(struct kvm_vcpu *vcpu);
+void kvm_nested_setup_mdcr_el2(struct kvm_vcpu *vcpu, u64 *mdcr);
struct kvm_s2_trans {
phys_addr_t output;
diff --git a/arch/arm64/kvm/debug.c b/arch/arm64/kvm/debug.c
index f4d7b12045e8f..89c9fb0cfeee2 100644
--- a/arch/arm64/kvm/debug.c
+++ b/arch/arm64/kvm/debug.c
@@ -37,38 +37,41 @@ static int cpu_has_spe(u64 dfr0)
*/
static void kvm_arm_setup_mdcr_el2(struct kvm_vcpu *vcpu)
{
+ u64 mdcr;
+
preempt_disable();
/*
* This also clears MDCR_EL2_E2PB_MASK and MDCR_EL2_E2TB_MASK
* to disable guest access to the profiling and trace buffers
*/
- vcpu->arch.mdcr_el2 = FIELD_PREP(MDCR_EL2_HPMN,
- *host_data_ptr(nr_event_counters));
- vcpu->arch.mdcr_el2 |= (MDCR_EL2_TPM |
- MDCR_EL2_TPMS |
- MDCR_EL2_TTRF |
- MDCR_EL2_TPMCR |
- MDCR_EL2_TDRA |
- MDCR_EL2_TDOSA);
+ mdcr = FIELD_PREP(MDCR_EL2_HPMN, *host_data_ptr(nr_event_counters));
+ mdcr |= (MDCR_EL2_TPM |
+ MDCR_EL2_TPMS |
+ MDCR_EL2_TTRF |
+ MDCR_EL2_TPMCR |
+ MDCR_EL2_TDRA |
+ MDCR_EL2_TDOSA);
/* Is the VM being debugged by userspace? */
if (vcpu->guest_debug)
/* Route all software debug exceptions to EL2 */
- vcpu->arch.mdcr_el2 |= MDCR_EL2_TDE;
+ mdcr |= MDCR_EL2_TDE;
/*
* Trap debug registers if the guest doesn't have ownership of them.
*/
if (!kvm_guest_owns_debug_regs(vcpu))
- vcpu->arch.mdcr_el2 |= MDCR_EL2_TDA;
+ mdcr |= MDCR_EL2_TDA;
if (vcpu_has_nv(vcpu))
- kvm_nested_setup_mdcr_el2(vcpu);
+ kvm_nested_setup_mdcr_el2(vcpu, &mdcr);
+
+ WRITE_ONCE(vcpu->arch.mdcr_el2, mdcr);
/* Write MDCR_EL2 directly if we're already at EL2 */
if (has_vhe())
- write_sysreg(vcpu->arch.mdcr_el2, mdcr_el2);
+ write_sysreg(mdcr, mdcr_el2);
preempt_enable();
}
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index 4d83281a7a28b..b80d90ede9004 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -2237,12 +2237,12 @@ void kvm_nested_sync_hwstate(struct kvm_vcpu *vcpu)
MDCR_EL2_TDCC | \
MDCR_EL2_TDOSA)
-void kvm_nested_setup_mdcr_el2(struct kvm_vcpu *vcpu)
+void kvm_nested_setup_mdcr_el2(struct kvm_vcpu *vcpu, u64 *mdcr)
{
u64 guest_mdcr = __vcpu_sys_reg(vcpu, MDCR_EL2);
if (is_nested_ctxt(vcpu))
- vcpu->arch.mdcr_el2 |= (guest_mdcr & NV_MDCR_GUEST_INCLUDE);
+ *mdcr |= (guest_mdcr & NV_MDCR_GUEST_INCLUDE);
/*
* In yet another example where FEAT_NV2 is fscking broken, accesses
* to MDSCR_EL1 are redirected to the VNCR despite having an effect
@@ -2252,5 +2252,5 @@ void kvm_nested_setup_mdcr_el2(struct kvm_vcpu *vcpu)
* trap MDSCR_EL1.
*/
else if (!cpus_have_final_cap(ARM64_HAS_FGT))
- vcpu->arch.mdcr_el2 |= MDCR_EL2_TDA;
+ *mdcr |= MDCR_EL2_TDA;
}
--
2.56.1.708.g77570a6a1d-goog
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v10 16/33] KVM: arm64: Set up MDCR_EL2 to handle a Partitioned PMU
2026-11-19 18:51 [PATCH v10 00/33] ARM64 PMU Partitioning Colton Lewis
` (14 preceding siblings ...)
2026-11-19 18:51 ` [PATCH v10 15/33] KVM: arm64: Construct mdcr_el2 in a local variable in kvm_arm_setup_mdcr_el2() Colton Lewis
@ 2026-11-19 18:51 ` Colton Lewis
2026-11-19 19:21 ` sashiko-bot
2026-11-19 18:51 ` [PATCH v10 17/33] KVM: arm64: Context swap Partitioned PMU guest registers Colton Lewis
` (16 subsequent siblings)
32 siblings, 1 reply; 67+ messages in thread
From: Colton Lewis @ 2026-11-19 18:51 UTC (permalink / raw)
To: Marc Zyngier, Oliver Upton
Cc: Catalin Marinas, Will Deacon, Mark Rutland, Peter Zijlstra,
Ingo Molnar, Paolo Bonzini, Russell King, Jonathan Corbet,
Fuad Tabba, Joey Gouly, Suzuki K Poulose, Zenghui Yu, Shuah Khan,
James Clark, Alexandru Elisei, Mingwei Zhang, Ganapatrao Kulkarni,
kvmarm, linux-arm-kernel, kvm, linux-perf-users, linux-kselftest,
linux-doc, linux-kernel
Configure MDCR_EL2 for a Partitioned PMU in kvm_arm_setup_mdcr_el2():
set HPME, HPMD, and HCCD (plus HLP on FEAT_PMUv3p5 so host counters
retain 64-bit overflow width), program HPMN to the guest's general
purpose counter count, and clear coarse-grained TPM and TPMCR traps when
FEAT_FGT is available. Protect MDCR_EL2 updates with local_daif_save()
so GICv3 pseudo-NMI PMU handlers toggling MDCR_EL2.HPME cannot race with
kvm_arm_setup_mdcr_el2().
Update kvm_nested_setup_mdcr_el2() for the two nested execution contexts:
- Nested guest context (is_nested_ctxt(vcpu), L2 at vEL1/vEL0):
NV_MDCR_GUEST_INCLUDE already propagates L1's virtual TPM and TPMCR.
When L1 sub-partitions its counters (vHPMN < nr_pmu_counters, so
kvm_pmu_hyp_counter_mask(vcpu) is non-zero), physical MDCR_EL2.HPMN
still equals nr_pmu_counters, so hardware alone cannot stop L2 from
accessing L1's hypervisor counters (vHPMN..nr_pmu_counters-1) or
reading nr_pmu_counters from PMCR_EL0.N. Force TPM and TPMCR so L0
emulates 0..vHPMN-1 locally and forwards hypervisor counter traps to
L1.
- Nested hypervisor context (else branch, L1): under VHE, both L1 vEL2
(physical EL1) and VHE host EL0 (physical EL0) run in this context
without exiting to L0 across ERET and SVC. When L1's PMUSERENR_EL0 is
0, hardware traps EL0 PMU accesses directly to L1 vEL2, allowing TPM
and TPMCR to stay clear for untrapped L1 vEL2 access. When L1 enables
EL0 access in PMUSERENR_EL0, propagate L1's virtual TPM/TPMCR (and set
TPM/TPMCR if L1 sub-partitions counters) so VHE host EL0 cannot bypass
L1's traps or vHPMN boundary.
Signed-off-by: Colton Lewis <coltonlewis@google.com>
---
arch/arm64/kvm/debug.c | 41 +++++++++++++++++++++++++++++++++++++++--
arch/arm64/kvm/nested.c | 35 ++++++++++++++++++++++++-----------
2 files changed, 63 insertions(+), 13 deletions(-)
diff --git a/arch/arm64/kvm/debug.c b/arch/arm64/kvm/debug.c
index 89c9fb0cfeee2..a1cb4c076bdb4 100644
--- a/arch/arm64/kvm/debug.c
+++ b/arch/arm64/kvm/debug.c
@@ -11,6 +11,7 @@
#include <linux/hw_breakpoint.h>
#include <asm/arm_pmuv3.h>
+#include <asm/daifflags.h>
#include <asm/debug-monitors.h>
#include <asm/kvm_asm.h>
#include <asm/kvm_arm.h>
@@ -37,9 +38,10 @@ static int cpu_has_spe(u64 dfr0)
*/
static void kvm_arm_setup_mdcr_el2(struct kvm_vcpu *vcpu)
{
+ unsigned long daif;
u64 mdcr;
- preempt_disable();
+ daif = local_daif_save();
/*
* This also clears MDCR_EL2_E2PB_MASK and MDCR_EL2_E2TB_MASK
@@ -53,6 +55,41 @@ static void kvm_arm_setup_mdcr_el2(struct kvm_vcpu *vcpu)
MDCR_EL2_TDRA |
MDCR_EL2_TDOSA);
+ if (kvm_pmu_is_partitioned(vcpu->kvm)) {
+ u8 nr_host_cntr = *host_data_ptr(nr_event_counters);
+ u8 nr_guest_cntr = vcpu->kvm->arch.nr_pmu_counters;
+
+ if (cpumask_test_cpu(smp_processor_id(),
+ vcpu->kvm->arch.supported_cpus) &&
+ nr_guest_cntr < nr_host_cntr &&
+ (cpus_have_final_cap(ARM64_HAS_HPMN0) || nr_guest_cntr > 0)) {
+ u64 hpmn = FIELD_GET(MDCR_EL2_HPMN, read_sysreg(mdcr_el2));
+ bool hpme;
+
+ if (hpmn < nr_host_cntr)
+ hpme = !!(read_sysreg(mdcr_el2) & MDCR_EL2_HPME);
+ else
+ hpme = !!(read_sysreg(pmcr_el0) & ARMV8_PMU_PMCR_E);
+
+ mdcr |= (MDCR_EL2_HPMD | MDCR_EL2_HCCD);
+ if (is_pmuv3p5(read_pmuver()))
+ mdcr |= MDCR_EL2_HLP;
+ if (hpme)
+ mdcr |= MDCR_EL2_HPME;
+
+ /*
+ * Take out the coarse grain traps if we are using
+ * fine grain traps and enforce counter access with
+ * HPMN.
+ */
+ mdcr &= ~MDCR_EL2_HPMN;
+ mdcr |= FIELD_PREP(MDCR_EL2_HPMN, nr_guest_cntr);
+
+ if (cpus_have_final_cap(ARM64_HAS_FGT))
+ mdcr &= ~(MDCR_EL2_TPM | MDCR_EL2_TPMCR);
+ }
+ }
+
/* Is the VM being debugged by userspace? */
if (vcpu->guest_debug)
/* Route all software debug exceptions to EL2 */
@@ -73,7 +110,7 @@ static void kvm_arm_setup_mdcr_el2(struct kvm_vcpu *vcpu)
if (has_vhe())
write_sysreg(mdcr, mdcr_el2);
- preempt_enable();
+ local_daif_restore(daif);
}
void kvm_init_host_debug_data(void)
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index b80d90ede9004..06a535c54b708 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -2241,16 +2241,29 @@ void kvm_nested_setup_mdcr_el2(struct kvm_vcpu *vcpu, u64 *mdcr)
{
u64 guest_mdcr = __vcpu_sys_reg(vcpu, MDCR_EL2);
- if (is_nested_ctxt(vcpu))
+ if (is_nested_ctxt(vcpu)) {
*mdcr |= (guest_mdcr & NV_MDCR_GUEST_INCLUDE);
- /*
- * In yet another example where FEAT_NV2 is fscking broken, accesses
- * to MDSCR_EL1 are redirected to the VNCR despite having an effect
- * at EL2. Use a big hammer to apply sanity.
- *
- * Unless of course we have FEAT_FGT, in which case we can precisely
- * trap MDSCR_EL1.
- */
- else if (!cpus_have_final_cap(ARM64_HAS_FGT))
- *mdcr |= MDCR_EL2_TDA;
+ if (kvm_pmu_is_partitioned(vcpu->kvm) &&
+ kvm_pmu_hyp_counter_mask(vcpu))
+ *mdcr |= MDCR_EL2_TPM | MDCR_EL2_TPMCR;
+ } else {
+ /*
+ * In yet another example where FEAT_NV2 is fscking broken,
+ * accesses to MDSCR_EL1 are redirected to the VNCR despite
+ * having an effect at EL2. Use a big hammer to apply sanity.
+ *
+ * Unless of course we have FEAT_FGT, in which case we can
+ * precisely trap MDSCR_EL1.
+ */
+ if (!cpus_have_final_cap(ARM64_HAS_FGT))
+ *mdcr |= MDCR_EL2_TDA;
+
+ if (kvm_pmu_is_partitioned(vcpu->kvm) &&
+ (__vcpu_sys_reg(vcpu, PMUSERENR_EL0) &
+ ARMV8_PMU_USERENR_MASK)) {
+ *mdcr |= guest_mdcr & (MDCR_EL2_TPM | MDCR_EL2_TPMCR);
+ if (kvm_pmu_hyp_counter_mask(vcpu))
+ *mdcr |= MDCR_EL2_TPM | MDCR_EL2_TPMCR;
+ }
+ }
}
--
2.56.1.708.g77570a6a1d-goog
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v10 17/33] KVM: arm64: Context swap Partitioned PMU guest registers
2026-11-19 18:51 [PATCH v10 00/33] ARM64 PMU Partitioning Colton Lewis
` (15 preceding siblings ...)
2026-11-19 18:51 ` [PATCH v10 16/33] KVM: arm64: Set up MDCR_EL2 to handle a Partitioned PMU Colton Lewis
@ 2026-11-19 18:51 ` Colton Lewis
2026-11-19 19:17 ` sashiko-bot
2026-11-19 18:52 ` [PATCH v10 18/33] KVM: arm64: Enforce PMU event filter at vcpu_load() Colton Lewis
` (15 subsequent siblings)
32 siblings, 1 reply; 67+ messages in thread
From: Colton Lewis @ 2026-11-19 18:51 UTC (permalink / raw)
To: Marc Zyngier, Oliver Upton
Cc: Catalin Marinas, Will Deacon, Mark Rutland, Peter Zijlstra,
Ingo Molnar, Paolo Bonzini, Russell King, Jonathan Corbet,
Fuad Tabba, Joey Gouly, Suzuki K Poulose, Zenghui Yu, Shuah Khan,
James Clark, Alexandru Elisei, Mingwei Zhang, Ganapatrao Kulkarni,
kvmarm, linux-arm-kernel, kvm, linux-perf-users, linux-kselftest,
linux-doc, linux-kernel
Save and restore the guest PMU registers that are untrapped when the PMU
is partitioned:
- PMEVCNTRn_EL0
- PMCCNTR_EL0
- PMSELR_EL0 (when MDCR_EL2.TPM is clear)
- PMUSERENR_EL0 (when MDCR_EL2.TPM is clear)
- PMCR_EL0
- PMCNTENSET_EL0 / PMCNTENCLR_EL0
- PMINTENSET_EL1 / PMINTENCLR_EL1
Mask counter bitmaps to guest-reserved counters so host-owned counters
in HPMN..N-1 are left untouched. Clear hardware guest overflow flags on
load before enabling PMINTENSET_EL1 to avoid spurious interrupts, flush
pending hardware guest overflow flags into shadow PMOVSSET_EL0 on put,
and restore physical PMCR_EL0.E from MDCR_EL2.HPME on put so active host
perf events continue counting.
PMEVTYPER<n>_EL0 and PMCCFILTR_EL0 are loaded in the next patch after
applying the KVM PMU event filter. A subsequent patch makes these swaps
lazy until the guest enables the PMU.
Signed-off-by: Colton Lewis <coltonlewis@google.com>
---
arch/arm64/kvm/arm.c | 4 +-
arch/arm64/kvm/pmu-direct.c | 175 ++++++++++++++++++++++++++++++++++++
include/kvm/arm_pmu.h | 6 +-
3 files changed, 183 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 157bf4b812020..fc420d8f0e4fe 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -767,6 +767,7 @@ static void vhe_vcpu_load(struct kvm_vcpu *vcpu)
kvm_vcpu_load_fgt(vcpu);
kvm_vcpu_load_vhe(vcpu);
kvm_arch_vcpu_load_fp(vcpu);
+ kvm_pmu_load(vcpu);
kvm_vcpu_pmu_restore_guest(vcpu);
vcpu_load_pvtime(vcpu);
@@ -821,12 +822,13 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
static void vhe_vcpu_put(struct kvm_vcpu *vcpu)
{
+ kvm_pmu_put(vcpu);
+ kvm_vcpu_pmu_restore_host(vcpu);
kvm_vcpu_put_debug(vcpu);
kvm_arch_vcpu_put_fp(vcpu);
kvm_vcpu_put_vhe(vcpu);
kvm_timer_vcpu_put(vcpu);
kvm_vgic_put(vcpu);
- kvm_vcpu_pmu_restore_host(vcpu);
if (vcpu_has_nv(vcpu))
kvm_vcpu_put_hw_mmu(vcpu);
diff --git a/arch/arm64/kvm/pmu-direct.c b/arch/arm64/kvm/pmu-direct.c
index 13f5bcc2a9084..5c8a8c42e353d 100644
--- a/arch/arm64/kvm/pmu-direct.c
+++ b/arch/arm64/kvm/pmu-direct.c
@@ -344,3 +344,178 @@ u64 kvm_pmu_guest_counter_mask(void)
{
return kvm_vcpu_pmu_guest_counter_mask(kvm_get_running_vcpu());
}
+
+/**
+ * kvm_pmu_load() - Load untrapped PMU registers
+ * @vcpu: Pointer to struct kvm_vcpu
+ *
+ * Load all untrapped PMU registers from the VCPU into the PCPU. Mask
+ * to only bits belonging to guest-reserved counters and leave
+ * host-reserved counters alone in bitmask registers.
+ */
+void kvm_pmu_load(struct kvm_vcpu *vcpu)
+{
+ struct arm_pmu *pmu = vcpu->kvm->arch.arm_pmu;
+ unsigned long guest_counters;
+ unsigned long flags;
+ u64 mask;
+ u8 i;
+ u64 val;
+
+ /*
+ * If we aren't guest-owned then we know the guest isn't using
+ * the PMU anyway, so no need to bother with the swap.
+ */
+ if (!kvm_pmu_is_partitioned(vcpu->kvm) || !vcpu->wants_to_run || !pmu)
+ return;
+
+ local_irq_save(flags);
+ if (!cpumask_test_cpu(smp_processor_id(), &pmu->supported_cpus)) {
+ local_irq_restore(flags);
+ return;
+ }
+
+ guest_counters = kvm_vcpu_pmu_guest_counter_mask(vcpu);
+
+ for_each_set_bit(i, &guest_counters, ARMPMU_MAX_HWEVENTS) {
+ val = __vcpu_sys_reg(vcpu, PMEVCNTR0_EL0 + i);
+
+ if (i == ARMV8_PMU_CYCLE_IDX)
+ write_pmccntr(val);
+ else
+ write_pmevcntrn(i, val);
+ }
+
+ if (!(vcpu->arch.mdcr_el2 & MDCR_EL2_TPM)) {
+ val = __vcpu_sys_reg(vcpu, PMSELR_EL0);
+ write_sysreg(val, pmselr_el0);
+
+ val = __vcpu_sys_reg(vcpu, PMUSERENR_EL0) &
+ ARMV8_PMU_USERENR_MASK;
+ write_sysreg(val, pmuserenr_el0);
+ }
+
+ /* Save only the stateful writable bits. */
+ val = __vcpu_sys_reg(vcpu, PMCR_EL0);
+ mask = ARMV8_PMU_PMCR_MASK &
+ ~(ARMV8_PMU_PMCR_P | ARMV8_PMU_PMCR_C);
+ if (!kvm_has_feat(vcpu->kvm, ID_AA64DFR0_EL1, PMUVer, V3P5))
+ mask &= ~ARMV8_PMU_PMCR_LP;
+ write_sysreg(val & mask, pmcr_el0);
+
+ /*
+ * When handling these:
+ * 1. Apply only the bits for guest counters (indicated by mask)
+ * 2. Use the different registers for set and clear
+ */
+ mask = guest_counters;
+
+ /*
+ * Clear the hardware overflow flags so there is no chance of
+ * creating spurious interrupts. The hardware here is never
+ * the canonical version anyway.
+ */
+ write_sysreg(mask, pmovsclr_el0);
+ isb();
+
+ val = __vcpu_sys_reg(vcpu, PMCNTENSET_EL0);
+ write_sysreg(val & mask, pmcntenset_el0);
+ write_sysreg(~val & mask, pmcntenclr_el0);
+
+ val = __vcpu_sys_reg(vcpu, PMINTENSET_EL1);
+ write_sysreg(val & mask, pmintenset_el1);
+ write_sysreg(~val & mask, pmintenclr_el1);
+
+ local_irq_restore(flags);
+}
+
+/**
+ * kvm_pmu_put() - Put untrapped PMU registers
+ * @vcpu: Pointer to struct kvm_vcpu
+ *
+ * Put all untrapped PMU registers from the PCPU into the VCPU. Mask
+ * to only bits belonging to guest-reserved counters and leave
+ * host-reserved counters alone in bitmask registers.
+ */
+void kvm_pmu_put(struct kvm_vcpu *vcpu)
+{
+ struct arm_pmu *pmu = vcpu->kvm->arch.arm_pmu;
+ unsigned long guest_counters;
+ unsigned long flags;
+ u64 pmcr_mask;
+ u64 mask;
+ u8 i;
+ u64 val;
+
+ /*
+ * If we aren't guest-owned then we know the guest is not
+ * accessing the PMU anyway, so no need to bother with the
+ * swap.
+ */
+ if (!kvm_pmu_is_partitioned(vcpu->kvm) || !vcpu->wants_to_run || !pmu)
+ return;
+
+ local_irq_save(flags);
+ if (!cpumask_test_cpu(smp_processor_id(), &pmu->supported_cpus)) {
+ local_irq_restore(flags);
+ return;
+ }
+
+ guest_counters = kvm_vcpu_pmu_guest_counter_mask(vcpu);
+ mask = guest_counters;
+
+ /* Mask these to only save the guest relevant bits. */
+ val = read_sysreg(pmcntenset_el0);
+ __vcpu_assign_sys_reg(vcpu, PMCNTENSET_EL0, val & mask);
+
+ val = read_sysreg(pmintenset_el1);
+ __vcpu_assign_sys_reg(vcpu, PMINTENSET_EL1, val & mask);
+
+ /* Stop guest counters and disable interrupts in hardware first. */
+ write_sysreg(mask, pmcntenclr_el0);
+ write_sysreg(mask, pmintenclr_el1);
+ isb();
+
+ for_each_set_bit(i, &guest_counters, ARMPMU_MAX_HWEVENTS) {
+ if (i == ARMV8_PMU_CYCLE_IDX) {
+ val = read_pmccntr();
+ write_pmccntr(0);
+ } else {
+ val = read_pmevcntrn(i);
+ write_pmevcntrn(i, 0);
+ }
+
+ __vcpu_assign_sys_reg(vcpu, PMEVCNTR0_EL0 + i, val);
+ }
+
+ if (!(vcpu->arch.mdcr_el2 & MDCR_EL2_TPM)) {
+ val = read_sysreg(pmselr_el0);
+ __vcpu_assign_sys_reg(vcpu, PMSELR_EL0, val);
+ write_sysreg(0, pmselr_el0);
+
+ val = read_sysreg(pmuserenr_el0) & ARMV8_PMU_USERENR_MASK;
+ __vcpu_assign_sys_reg(vcpu, PMUSERENR_EL0, val);
+ write_sysreg(ARMV8_PMU_USERENR_MASK, pmuserenr_el0);
+ }
+
+ val = read_sysreg(pmcr_el0);
+ pmcr_mask = ARMV8_PMU_PMCR_MASK;
+ if (!kvm_has_feat(vcpu->kvm, ID_AA64DFR0_EL1, PMUVer, V3P5))
+ pmcr_mask &= ~ARMV8_PMU_PMCR_LP;
+ __vcpu_rmw_sys_reg(vcpu, PMCR_EL0, &=, ~ARMV8_PMU_PMCR_MASK);
+ __vcpu_rmw_sys_reg(vcpu, PMCR_EL0, |=, val & pmcr_mask);
+
+ val = ARMV8_PMU_PMCR_LC;
+ if (pmu->pmuver >= ID_AA64DFR0_EL1_PMUVer_V3P5)
+ val |= ARMV8_PMU_PMCR_LP;
+ if (vcpu->arch.mdcr_el2 & MDCR_EL2_HPME)
+ val |= ARMV8_PMU_PMCR_E;
+ write_sysreg(val, pmcr_el0);
+
+ /* Save pending guest hardware overflows. */
+ val = read_sysreg(pmovsset_el0);
+ __vcpu_rmw_sys_reg(vcpu, PMOVSSET_EL0, |=, val & mask);
+ write_sysreg(val & mask, pmovsclr_el0);
+
+ local_irq_restore(flags);
+}
diff --git a/include/kvm/arm_pmu.h b/include/kvm/arm_pmu.h
index 276ab74fa0208..4580f62234128 100644
--- a/include/kvm/arm_pmu.h
+++ b/include/kvm/arm_pmu.h
@@ -102,6 +102,9 @@ u64 kvm_pmu_direct_reg_read(struct kvm_vcpu *vcpu, unsigned int reg);
void kvm_pmu_direct_reg_write(struct kvm_vcpu *vcpu, unsigned int reg,
u64 val, bool set);
u64 kvm_pmu_guest_counter_mask(void);
+void kvm_pmu_load(struct kvm_vcpu *vcpu);
+void kvm_pmu_put(struct kvm_vcpu *vcpu);
+
/*
* Updates the vcpu's view of the pmu events for this cpu.
* Must be called before every vcpu run after disabling interrupts, to ensure
@@ -146,10 +149,11 @@ static inline u64 kvm_pmu_direct_reg_read(struct kvm_vcpu *vcpu,
{
return 0;
}
-
static inline void kvm_pmu_direct_reg_write(struct kvm_vcpu *vcpu,
unsigned int reg, u64 val,
bool set) {}
+static inline void kvm_pmu_load(struct kvm_vcpu *vcpu) {}
+static inline void kvm_pmu_put(struct kvm_vcpu *vcpu) {}
static inline void kvm_pmu_set_counter_value(struct kvm_vcpu *vcpu,
u64 select_idx, u64 val) {}
static inline void kvm_pmu_set_counter_value_user(struct kvm_vcpu *vcpu,
--
2.56.1.708.g77570a6a1d-goog
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v10 18/33] KVM: arm64: Enforce PMU event filter at vcpu_load()
2026-11-19 18:51 [PATCH v10 00/33] ARM64 PMU Partitioning Colton Lewis
` (16 preceding siblings ...)
2026-11-19 18:51 ` [PATCH v10 17/33] KVM: arm64: Context swap Partitioned PMU guest registers Colton Lewis
@ 2026-11-19 18:52 ` Colton Lewis
2026-11-19 19:14 ` sashiko-bot
2026-11-19 18:52 ` [PATCH v10 19/33] perf: Add perf_pmu_resched_update() Colton Lewis
` (14 subsequent siblings)
32 siblings, 1 reply; 67+ messages in thread
From: Colton Lewis @ 2026-11-19 18:52 UTC (permalink / raw)
To: Marc Zyngier, Oliver Upton
Cc: Catalin Marinas, Will Deacon, Mark Rutland, Peter Zijlstra,
Ingo Molnar, Paolo Bonzini, Russell King, Jonathan Corbet,
Fuad Tabba, Joey Gouly, Suzuki K Poulose, Zenghui Yu, Shuah Khan,
James Clark, Alexandru Elisei, Mingwei Zhang, Ganapatrao Kulkarni,
kvmarm, linux-arm-kernel, kvm, linux-perf-users, linux-kselftest,
linux-doc, linux-kernel
The KVM PMU event filter uAPI requires disallowed events not to count.
Because userspace can install or update the event filter independently
of guest register writes, re-apply the filter on every vCPU load as well
as on trapped PMEVTYPER<n>_EL0 and PMCCFILTR_EL0 writes, excluding all
exception levels in hardware whenever the selected event is blocked.
When running in a nested hypervisor context (is_hyp_ctxt(vcpu)), L1
vEL2 executes at physical NS-EL1: remap guest ARMV8_PMU_INCLUDE_EL2 onto
physical EL1 (clearing ARMV8_PMU_EXCLUDE_NS_EL1 so P == NSK) and honor
guest MDCR_EL2.HPMD for event counters 0..HPMN-1 and MDCR_EL2.HCCD for
PMCCNTR_EL0.
Signed-off-by: Colton Lewis <coltonlewis@google.com>
---
arch/arm64/kvm/pmu-direct.c | 78 +++++++++++++++++++++++++++++++++++++
1 file changed, 78 insertions(+)
diff --git a/arch/arm64/kvm/pmu-direct.c b/arch/arm64/kvm/pmu-direct.c
index 5c8a8c42e353d..ace3e8e781538 100644
--- a/arch/arm64/kvm/pmu-direct.c
+++ b/arch/arm64/kvm/pmu-direct.c
@@ -242,6 +242,67 @@ u64 kvm_pmu_direct_reg_read(struct kvm_vcpu *vcpu, unsigned int reg)
}
}
+static void kvm_pmu_apply_single_event_filter(struct kvm_vcpu *vcpu, u8 idx)
+{
+ u64 guest_counters;
+ u64 evtyper_set = ARMV8_PMU_EXCLUDE_EL0 |
+ ARMV8_PMU_EXCLUDE_EL1;
+ u64 evtyper_clr = ARMV8_PMU_INCLUDE_EL2;
+ bool guest_include_el2;
+ u64 val;
+ u64 evsel;
+
+ guard(preempt)();
+
+ if (!kvm_pmu_part_on_cpu(vcpu))
+ return;
+
+ guest_counters = kvm_vcpu_pmu_guest_counter_mask(vcpu);
+ if (!test_bit(idx, (unsigned long *)&guest_counters))
+ return;
+
+ if (idx == ARMV8_PMU_CYCLE_IDX) {
+ val = __vcpu_sys_reg(vcpu, PMCCFILTR_EL0);
+ val &= kvm_pmu_evtyper_mask(vcpu->kvm) &
+ ~kvm_pmu_event_mask(vcpu->kvm);
+ evsel = ARMV8_PMUV3_PERFCTR_CPU_CYCLES;
+ } else {
+ val = __vcpu_sys_reg(vcpu, PMEVTYPER0_EL0 + idx);
+ val &= kvm_pmu_evtyper_mask(vcpu->kvm);
+ evsel = val & kvm_pmu_event_mask(vcpu->kvm);
+ }
+
+ guest_include_el2 = (val & ARMV8_PMU_INCLUDE_EL2);
+ val &= ~evtyper_clr;
+
+ if (unlikely(is_hyp_ctxt(vcpu))) {
+ u64 mdcr = __vcpu_sys_reg(vcpu, MDCR_EL2);
+ bool hpmd = (idx == ARMV8_PMU_CYCLE_IDX) ?
+ (mdcr & MDCR_EL2_HCCD) :
+ (!kvm_pmu_counter_is_hyp(vcpu, idx) &&
+ (mdcr & MDCR_EL2_HPMD));
+
+ val &= ~ARMV8_PMU_EXCLUDE_NS_EL1;
+ if (guest_include_el2 && !hpmd)
+ val &= ~ARMV8_PMU_EXCLUDE_EL1;
+ else
+ val |= ARMV8_PMU_EXCLUDE_EL1;
+ }
+
+ if (vcpu->kvm->arch.pmu_filter &&
+ evsel != ARMV8_PMUV3_PERFCTR_SW_INCR &&
+ evsel != ARMV8_PMUV3_PERFCTR_CHAIN &&
+ !test_bit(evsel, vcpu->kvm->arch.pmu_filter)) {
+ val |= evtyper_set;
+ val &= ~(ARMV8_PMU_EXCLUDE_NS_EL0 | ARMV8_PMU_EXCLUDE_NS_EL1);
+ }
+
+ if (idx == ARMV8_PMU_CYCLE_IDX)
+ write_pmccfiltr(val);
+ else
+ write_pmevtypern(idx, val);
+}
+
/**
* kvm_pmu_direct_reg_write() - Write a guest PMU register for Partitioned PMU
* @vcpu: Pointer to vcpu struct
@@ -281,6 +342,7 @@ void kvm_pmu_direct_reg_write(struct kvm_vcpu *vcpu, unsigned int reg,
if (reg == PMCCFILTR_EL0)
mask &= ~kvm_pmu_event_mask(vcpu->kvm);
__vcpu_assign_sys_reg(vcpu, reg, val & mask);
+ kvm_pmu_apply_single_event_filter(vcpu, reg - PMEVTYPER0_EL0);
break;
case PMCNTENSET_EL0:
val &= kvm_vcpu_pmu_guest_counter_mask(vcpu);
@@ -345,6 +407,21 @@ u64 kvm_pmu_guest_counter_mask(void)
return kvm_vcpu_pmu_guest_counter_mask(kvm_get_running_vcpu());
}
+static void kvm_pmu_apply_event_filter(struct kvm_vcpu *vcpu)
+{
+ struct arm_pmu *pmu = vcpu->kvm->arch.arm_pmu;
+ unsigned long guest_counters;
+ u8 i;
+
+ if (!pmu)
+ return;
+
+ guest_counters = kvm_vcpu_pmu_guest_counter_mask(vcpu);
+
+ for_each_set_bit(i, &guest_counters, ARMPMU_MAX_HWEVENTS)
+ kvm_pmu_apply_single_event_filter(vcpu, i);
+}
+
/**
* kvm_pmu_load() - Load untrapped PMU registers
* @vcpu: Pointer to struct kvm_vcpu
@@ -376,6 +453,7 @@ void kvm_pmu_load(struct kvm_vcpu *vcpu)
}
guest_counters = kvm_vcpu_pmu_guest_counter_mask(vcpu);
+ kvm_pmu_apply_event_filter(vcpu);
for_each_set_bit(i, &guest_counters, ARMPMU_MAX_HWEVENTS) {
val = __vcpu_sys_reg(vcpu, PMEVCNTR0_EL0 + i);
--
2.56.1.708.g77570a6a1d-goog
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v10 19/33] perf: Add perf_pmu_resched_update()
2026-11-19 18:51 [PATCH v10 00/33] ARM64 PMU Partitioning Colton Lewis
` (17 preceding siblings ...)
2026-11-19 18:52 ` [PATCH v10 18/33] KVM: arm64: Enforce PMU event filter at vcpu_load() Colton Lewis
@ 2026-11-19 18:52 ` Colton Lewis
2026-11-19 19:02 ` sashiko-bot
2026-11-19 18:52 ` [PATCH v10 20/33] perf/core: Prioritize kernel-owned pinned events when scheduling groups Colton Lewis
` (13 subsequent siblings)
32 siblings, 1 reply; 67+ messages in thread
From: Colton Lewis @ 2026-11-19 18:52 UTC (permalink / raw)
To: Marc Zyngier, Oliver Upton
Cc: Catalin Marinas, Will Deacon, Mark Rutland, Peter Zijlstra,
Ingo Molnar, Paolo Bonzini, Russell King, Jonathan Corbet,
Fuad Tabba, Joey Gouly, Suzuki K Poulose, Zenghui Yu, Shuah Khan,
James Clark, Alexandru Elisei, Mingwei Zhang, Ganapatrao Kulkarni,
kvmarm, linux-arm-kernel, kvm, linux-perf-users, linux-kselftest,
linux-doc, linux-kernel
To modify PMU guest counter reservations dynamically, the per-CPU
available counter mask must be updated while host perf events are
temporarily scheduled out.
Introduce perf_pmu_resched_update() to update PMU state while holding
cpuctx->ctx.lock in between scheduling perf events out and scheduling
them back in. It accepts a callback invoked between ctx_sched_out() and
perf_event_sched_in(), accomplishing atomic counter mask updates with
minimal perf API expansion.
Refactor ctx_resched() into __ctx_resched() to invoke the callback at
the appropriate point.
Tested-by: James Clark <james.clark@linaro.org>
Signed-off-by: Colton Lewis <coltonlewis@google.com>
---
include/linux/perf_event.h | 3 +++
kernel/events/core.c | 33 ++++++++++++++++++++++++++++++---
2 files changed, 33 insertions(+), 3 deletions(-)
diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h
index e1e11500df097..8140457939658 100644
--- a/include/linux/perf_event.h
+++ b/include/linux/perf_event.h
@@ -1242,6 +1242,9 @@ extern int perf_event_task_disable(void);
extern int perf_event_task_enable(void);
extern void perf_pmu_resched(struct pmu *pmu);
+extern void perf_pmu_resched_update(struct pmu *pmu,
+ void (*update)(struct pmu *, void *),
+ void *data);
extern int perf_event_refresh(struct perf_event *event, int refresh);
extern void perf_event_update_userpage(struct perf_event *event);
diff --git a/kernel/events/core.c b/kernel/events/core.c
index e870f12c3ad81..dbc42ded5f698 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -3010,9 +3010,10 @@ static void perf_event_sched_in(struct perf_cpu_context *cpuctx,
* event_type is a bit mask of the types of events involved. For CPU events,
* event_type is only either EVENT_PINNED or EVENT_FLEXIBLE.
*/
-static void ctx_resched(struct perf_cpu_context *cpuctx,
- struct perf_event_context *task_ctx,
- struct pmu *pmu, enum event_type_t event_type)
+static void __ctx_resched(struct perf_cpu_context *cpuctx,
+ struct perf_event_context *task_ctx,
+ struct pmu *pmu, enum event_type_t event_type,
+ void (*update)(struct pmu *, void *), void *data)
{
bool cpu_event = !!(event_type & EVENT_CPU);
struct perf_event_pmu_context *epc;
@@ -3048,6 +3049,9 @@ static void ctx_resched(struct perf_cpu_context *cpuctx,
else if (event_type & EVENT_PINNED)
ctx_sched_out(&cpuctx->ctx, pmu, EVENT_FLEXIBLE);
+ if (update)
+ update(pmu, data);
+
perf_event_sched_in(cpuctx, task_ctx, pmu, 0);
for_each_epc(epc, &cpuctx->ctx, pmu, 0)
@@ -3059,6 +3063,29 @@ static void ctx_resched(struct perf_cpu_context *cpuctx,
}
}
+static void ctx_resched(struct perf_cpu_context *cpuctx,
+ struct perf_event_context *task_ctx,
+ struct pmu *pmu, enum event_type_t event_type)
+{
+ __ctx_resched(cpuctx, task_ctx, pmu, event_type, NULL, NULL);
+}
+
+void perf_pmu_resched_update(struct pmu *pmu, void (*update)(struct pmu *, void *), void *data)
+{
+ struct perf_cpu_context *cpuctx;
+ struct perf_event_context *task_ctx;
+ unsigned long flags;
+
+ local_irq_save(flags);
+ cpuctx = this_cpu_ptr(&perf_cpu_context);
+ task_ctx = cpuctx->task_ctx;
+ perf_ctx_lock(cpuctx, task_ctx);
+ __ctx_resched(cpuctx, task_ctx, pmu, EVENT_ALL | EVENT_CPU, update, data);
+ perf_ctx_unlock(cpuctx, task_ctx);
+ local_irq_restore(flags);
+}
+EXPORT_SYMBOL_GPL(perf_pmu_resched_update);
+
void perf_pmu_resched(struct pmu *pmu)
{
struct perf_cpu_context *cpuctx = this_cpu_ptr(&perf_cpu_context);
--
2.56.1.708.g77570a6a1d-goog
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v10 20/33] perf/core: Prioritize kernel-owned pinned events when scheduling groups
2026-11-19 18:51 [PATCH v10 00/33] ARM64 PMU Partitioning Colton Lewis
` (18 preceding siblings ...)
2026-11-19 18:52 ` [PATCH v10 19/33] perf: Add perf_pmu_resched_update() Colton Lewis
@ 2026-11-19 18:52 ` Colton Lewis
2026-11-19 19:01 ` sashiko-bot
2026-11-19 18:52 ` [PATCH v10 21/33] KVM: arm64: Allow kvm_vcpu_pmu_resync_el0() to resync filters in process context Colton Lewis
` (12 subsequent siblings)
32 siblings, 1 reply; 67+ messages in thread
From: Colton Lewis @ 2026-11-19 18:52 UTC (permalink / raw)
To: Marc Zyngier, Oliver Upton
Cc: Catalin Marinas, Will Deacon, Mark Rutland, Peter Zijlstra,
Ingo Molnar, Paolo Bonzini, Russell King, Jonathan Corbet,
Fuad Tabba, Joey Gouly, Suzuki K Poulose, Zenghui Yu, Shuah Khan,
James Clark, Alexandru Elisei, Mingwei Zhang, Ganapatrao Kulkarni,
kvmarm, linux-arm-kernel, kvm, linux-perf-users, linux-kselftest,
linux-doc, linux-kernel
When scheduling event groups in visit_groups_merge(), pinned and
flexible groups are ordered by group_index (creation order). Normally,
kernel-owned pinned events such as the hardlockup NMI watchdog are
created at boot with group_index 0 and are scheduled ahead of userspace
events.
However, if the NMI watchdog is disabled and re-enabled at runtime
after a userspace pinned per-CPU event is already active, the
watchdog's group_index is larger than the userspace event's
group_index. When hardware counters are constrained (for example, when
PMU partitioning reserves counters for a guest and leaves only one
general-purpose counter for the host), the userspace pinned event
claims the remaining counter first and merge_sched_in() transitions the
kernel watchdog event into PERF_EVENT_STATE_ERROR.
Order pinned kernel-owned events (is_pinned_kernel_event()) ahead of
userspace pinned events in perf_event_groups_cmp() and
perf_less_group_idx() while leaving flexible groups ordered by
group_index for normal round-robin rotation.
Suggested-by: James Clark <james.clark@linaro.org>
Signed-off-by: Colton Lewis <coltonlewis@google.com>
---
kernel/events/core.c | 28 +++++++++++++++++++++-------
1 file changed, 21 insertions(+), 7 deletions(-)
diff --git a/kernel/events/core.c b/kernel/events/core.c
index dbc42ded5f698..6b82029b8ebc5 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -237,11 +237,16 @@ class_perf_ctx_lock_constructor(struct perf_cpu_context *cpuctx,
#define TASK_TOMBSTONE ((void *)-1L)
-static bool is_kernel_event(struct perf_event *event)
+static bool is_kernel_event(const struct perf_event *event)
{
return READ_ONCE(event->owner) == TASK_TOMBSTONE;
}
+static bool is_pinned_kernel_event(const struct perf_event *event)
+{
+ return event->attr.pinned && is_kernel_event(event);
+}
+
static DEFINE_PER_CPU(struct perf_cpu_context, perf_cpu_context);
struct perf_event_context *perf_cpu_task_ctx(void)
@@ -1781,7 +1786,8 @@ static inline struct cgroup *event_cgroup(const struct perf_event *event)
*/
static __always_inline int
perf_event_groups_cmp(const int left_cpu, const struct pmu *left_pmu,
- const struct cgroup *left_cgroup, const u64 left_group_index,
+ const struct cgroup *left_cgroup, const bool left_kernel,
+ const u64 left_group_index,
const struct perf_event *right)
{
if (left_cpu < right->cpu)
@@ -1824,6 +1830,9 @@ perf_event_groups_cmp(const int left_cpu, const struct pmu *left_pmu,
}
#endif
+ if (left_kernel != is_pinned_kernel_event(right))
+ return left_kernel ? -1 : 1;
+
if (left_group_index < right->group_index)
return -1;
if (left_group_index > right->group_index)
@@ -1839,7 +1848,8 @@ static inline bool __group_less(struct rb_node *a, const struct rb_node *b)
{
struct perf_event *e = __node_2_pe(a);
return perf_event_groups_cmp(e->cpu, e->pmu_ctx->pmu, event_cgroup(e),
- e->group_index, __node_2_pe(b)) < 0;
+ is_pinned_kernel_event(e), e->group_index,
+ __node_2_pe(b)) < 0;
}
struct __group_key {
@@ -1853,8 +1863,9 @@ static inline int __group_cmp(const void *key, const struct rb_node *node)
const struct __group_key *a = key;
const struct perf_event *b = __node_2_pe(node);
- /* partial/subtree match: @cpu, @pmu, @cgroup; ignore: @group_index */
- return perf_event_groups_cmp(a->cpu, a->pmu, a->cgroup, b->group_index, b);
+ /* partial/subtree match: @cpu, @pmu, @cgroup; ignore: @kernel, @group_index */
+ return perf_event_groups_cmp(a->cpu, a->pmu, a->cgroup,
+ is_pinned_kernel_event(b), b->group_index, b);
}
static inline int
@@ -1863,9 +1874,9 @@ __group_cmp_ignore_cgroup(const void *key, const struct rb_node *node)
const struct __group_key *a = key;
const struct perf_event *b = __node_2_pe(node);
- /* partial/subtree match: @cpu, @pmu, ignore: @cgroup, @group_index */
+ /* partial/subtree match: @cpu, @pmu, ignore: @cgroup, @kernel, @group_index */
return perf_event_groups_cmp(a->cpu, a->pmu, event_cgroup(b),
- b->group_index, b);
+ is_pinned_kernel_event(b), b->group_index, b);
}
/*
@@ -4024,6 +4035,9 @@ static bool perf_less_group_idx(const void *l, const void *r, void __always_unus
const struct perf_event *le = *(const struct perf_event **)l;
const struct perf_event *re = *(const struct perf_event **)r;
+ if (is_pinned_kernel_event(le) != is_pinned_kernel_event(re))
+ return is_pinned_kernel_event(le);
+
return le->group_index < re->group_index;
}
--
2.56.1.708.g77570a6a1d-goog
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v10 21/33] KVM: arm64: Allow kvm_vcpu_pmu_resync_el0() to resync filters in process context
2026-11-19 18:51 [PATCH v10 00/33] ARM64 PMU Partitioning Colton Lewis
` (19 preceding siblings ...)
2026-11-19 18:52 ` [PATCH v10 20/33] perf/core: Prioritize kernel-owned pinned events when scheduling groups Colton Lewis
@ 2026-11-19 18:52 ` Colton Lewis
2026-11-19 19:07 ` sashiko-bot
2026-11-19 18:52 ` [PATCH v10 22/33] KVM: arm64: Apply dynamic guest counter reservations Colton Lewis
` (11 subsequent siblings)
32 siblings, 1 reply; 67+ messages in thread
From: Colton Lewis @ 2026-11-19 18:52 UTC (permalink / raw)
To: Marc Zyngier, Oliver Upton
Cc: Catalin Marinas, Will Deacon, Mark Rutland, Peter Zijlstra,
Ingo Molnar, Paolo Bonzini, Russell King, Jonathan Corbet,
Fuad Tabba, Joey Gouly, Suzuki K Poulose, Zenghui Yu, Shuah Khan,
James Clark, Alexandru Elisei, Mingwei Zhang, Ganapatrao Kulkarni,
kvmarm, linux-arm-kernel, kvm, linux-perf-users, linux-kselftest,
linux-doc, linux-kernel
Previously, kvm_vcpu_pmu_resync_el0() only scheduled
KVM_REQ_RESYNC_PMU_EL0 when called from interrupt context (such as an
IPI reprogramming host perf events). When perf_pmu_resched_update() is
invoked synchronously from process context during vCPU load/put to
reschedule host events around guest-reserved counters,
armv8pmu_enable_event() calls kvm_set_pmu_events() and
kvm_vcpu_pmu_resync_el0() in process context.
Update kvm_vcpu_pmu_resync_el0() to call kvm_vcpu_pmu_restore_guest()
directly when not in interrupt context so that newly rescheduled host
perf events have their EL0 exclusion filters applied immediately.
Tested-by: James Clark <james.clark@linaro.org>
Signed-off-by: Colton Lewis <coltonlewis@google.com>
---
arch/arm64/kvm/pmu.c | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/kvm/pmu.c b/arch/arm64/kvm/pmu.c
index dbbe219ac874f..d3fd438a57b09 100644
--- a/arch/arm64/kvm/pmu.c
+++ b/arch/arm64/kvm/pmu.c
@@ -213,14 +213,17 @@ void kvm_vcpu_pmu_resync_el0(void)
{
struct kvm_vcpu *vcpu;
- if (!has_vhe() || !in_interrupt())
+ if (!has_vhe())
return;
vcpu = kvm_get_running_vcpu();
if (!vcpu)
return;
- kvm_make_request(KVM_REQ_RESYNC_PMU_EL0, vcpu);
+ if (in_interrupt())
+ kvm_make_request(KVM_REQ_RESYNC_PMU_EL0, vcpu);
+ else
+ kvm_vcpu_pmu_restore_guest(vcpu);
}
void kvm_host_pmu_init(struct arm_pmu *pmu)
--
2.56.1.708.g77570a6a1d-goog
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v10 22/33] KVM: arm64: Apply dynamic guest counter reservations
2026-11-19 18:51 [PATCH v10 00/33] ARM64 PMU Partitioning Colton Lewis
` (20 preceding siblings ...)
2026-11-19 18:52 ` [PATCH v10 21/33] KVM: arm64: Allow kvm_vcpu_pmu_resync_el0() to resync filters in process context Colton Lewis
@ 2026-11-19 18:52 ` Colton Lewis
2026-11-19 19:21 ` sashiko-bot
2026-11-19 18:52 ` [PATCH v10 23/33] KVM: arm64: Implement lazy PMU context swaps Colton Lewis
` (10 subsequent siblings)
32 siblings, 1 reply; 67+ messages in thread
From: Colton Lewis @ 2026-11-19 18:52 UTC (permalink / raw)
To: Marc Zyngier, Oliver Upton
Cc: Catalin Marinas, Will Deacon, Mark Rutland, Peter Zijlstra,
Ingo Molnar, Paolo Bonzini, Russell King, Jonathan Corbet,
Fuad Tabba, Joey Gouly, Suzuki K Poulose, Zenghui Yu, Shuah Khan,
James Clark, Alexandru Elisei, Mingwei Zhang, Ganapatrao Kulkarni,
kvmarm, linux-arm-kernel, kvm, linux-perf-users, linux-kselftest,
linux-doc, linux-kernel
Reserve and release guest PMU counters dynamically during vCPU load and
put rather than statically at VM creation.
Add kvm_pmu_set_guest_counters() in arch/arm64/kvm/pmu-direct.c, called
from __kvm_pmu_load() and __kvm_pmu_put() with local interrupts
disabled. When the requested guest counter mask collides with active
host events in cpuc->used_mask (or when releasing counters after
squeezing a host event), invoke perf_pmu_resched_update() with
kvm_pmu_update_mask() to update per-CPU cpuc->cntr_mask between
scheduling host events out and back in; otherwise update cpuc->cntr_mask
directly. Warn once in armpmu_add() if a pinned host event fails counter
allocation while guest counters are reserved.
Skip releasing and re-reserving host counters across nested ERET and
nested exception transitions (IN_NESTED_ERET and IN_NESTED_EXCEPTION)
since the vCPU remains on the same physical CPU.
Signed-off-by: Colton Lewis <coltonlewis@google.com>
---
arch/arm64/kvm/pmu-direct.c | 118 ++++++++++++++++++++++++++++++-----
drivers/perf/arm_pmu.c | 9 ++-
include/linux/perf/arm_pmu.h | 1 +
3 files changed, 113 insertions(+), 15 deletions(-)
diff --git a/arch/arm64/kvm/pmu-direct.c b/arch/arm64/kvm/pmu-direct.c
index ace3e8e781538..cc4ff00d6f41e 100644
--- a/arch/arm64/kvm/pmu-direct.c
+++ b/arch/arm64/kvm/pmu-direct.c
@@ -154,6 +154,70 @@ static u64 kvm_pmu_direct_pmcr_read(struct kvm_vcpu *vcpu)
return u64_replace_bits(pmcr, n, ARMV8_PMU_PMCR_N);
}
+/* Callback to update counter mask between perf scheduling */
+static void kvm_pmu_update_mask(struct pmu *pmu, void *data)
+{
+ struct arm_pmu *arm_pmu = to_arm_pmu(pmu);
+ struct pmu_hw_events *cpuc = this_cpu_ptr(arm_pmu->hw_events);
+ unsigned long *new_mask = data;
+
+ bitmap_copy(cpuc->cntr_mask, new_mask, ARMPMU_MAX_HWEVENTS);
+}
+
+/**
+ * kvm_pmu_set_guest_counters() - Handle dynamic counter reservations
+ * @cpu_pmu: struct arm_pmu to potentially modify
+ * @guest_mask: new guest mask for the pmu
+ *
+ * Check if guest counters will interfere with current host events and
+ * call into perf_pmu_resched_update if a reschedule is required.
+ */
+static void kvm_pmu_set_guest_counters(struct arm_pmu *cpu_pmu, u64 guest_mask)
+{
+ struct pmu_hw_events *cpuc;
+ DECLARE_BITMAP(guest_bitmap, ARMPMU_MAX_HWEVENTS);
+ DECLARE_BITMAP(new_mask, ARMPMU_MAX_HWEVENTS);
+ unsigned long flags;
+ bool need_resched = false;
+
+ if (!cpu_pmu)
+ return;
+
+ local_irq_save(flags);
+ cpuc = this_cpu_ptr(cpu_pmu->hw_events);
+
+ bitmap_from_arr64(guest_bitmap, &guest_mask, ARMPMU_MAX_HWEVENTS);
+ bitmap_copy(new_mask, cpu_pmu->cntr_mask, ARMPMU_MAX_HWEVENTS);
+
+ if (guest_mask) {
+ /* Subtract guest counters from available host mask */
+ bitmap_andnot(new_mask, new_mask, guest_bitmap, ARMPMU_MAX_HWEVENTS);
+
+ /* Did we collide with an active host event? */
+ if (bitmap_intersects(cpuc->used_mask, guest_bitmap, ARMPMU_MAX_HWEVENTS)) {
+ need_resched = true;
+ cpuc->host_squeezed = true;
+ }
+ } else {
+ /*
+ * Restoring to full mask.
+ * Only resched if we previously squeezed an event.
+ */
+ if (cpuc->host_squeezed) {
+ need_resched = true;
+ cpuc->host_squeezed = false;
+ }
+ }
+
+ if (need_resched)
+ /* Collision: run full perf reschedule */
+ perf_pmu_resched_update(&cpu_pmu->pmu, kvm_pmu_update_mask, new_mask);
+ else
+ /* Host was never using guest counters anyway */
+ bitmap_copy(cpuc->cntr_mask, new_mask, ARMPMU_MAX_HWEVENTS);
+ local_irq_restore(flags);
+}
+
static u64 kvm_pmu_direct_cntr_read(struct kvm_vcpu *vcpu, u8 idx)
{
lockdep_assert_preemption_disabled();
@@ -422,15 +486,7 @@ static void kvm_pmu_apply_event_filter(struct kvm_vcpu *vcpu)
kvm_pmu_apply_single_event_filter(vcpu, i);
}
-/**
- * kvm_pmu_load() - Load untrapped PMU registers
- * @vcpu: Pointer to struct kvm_vcpu
- *
- * Load all untrapped PMU registers from the VCPU into the PCPU. Mask
- * to only bits belonging to guest-reserved counters and leave
- * host-reserved counters alone in bitmask registers.
- */
-void kvm_pmu_load(struct kvm_vcpu *vcpu)
+static void __kvm_pmu_load(struct kvm_vcpu *vcpu, bool reserve_counters)
{
struct arm_pmu *pmu = vcpu->kvm->arch.arm_pmu;
unsigned long guest_counters;
@@ -446,13 +502,18 @@ void kvm_pmu_load(struct kvm_vcpu *vcpu)
if (!kvm_pmu_is_partitioned(vcpu->kvm) || !vcpu->wants_to_run || !pmu)
return;
+ preempt_disable();
local_irq_save(flags);
if (!cpumask_test_cpu(smp_processor_id(), &pmu->supported_cpus)) {
local_irq_restore(flags);
+ preempt_enable();
return;
}
guest_counters = kvm_vcpu_pmu_guest_counter_mask(vcpu);
+ if (reserve_counters)
+ kvm_pmu_set_guest_counters(pmu, guest_counters);
+
kvm_pmu_apply_event_filter(vcpu);
for_each_set_bit(i, &guest_counters, ARMPMU_MAX_HWEVENTS) {
@@ -505,17 +566,26 @@ void kvm_pmu_load(struct kvm_vcpu *vcpu)
write_sysreg(~val & mask, pmintenclr_el1);
local_irq_restore(flags);
+ preempt_enable();
}
/**
- * kvm_pmu_put() - Put untrapped PMU registers
+ * kvm_pmu_load() - Load untrapped PMU registers
* @vcpu: Pointer to struct kvm_vcpu
*
- * Put all untrapped PMU registers from the PCPU into the VCPU. Mask
+ * Load all untrapped PMU registers from the VCPU into the PCPU. Mask
* to only bits belonging to guest-reserved counters and leave
* host-reserved counters alone in bitmask registers.
*/
-void kvm_pmu_put(struct kvm_vcpu *vcpu)
+void kvm_pmu_load(struct kvm_vcpu *vcpu)
+{
+ bool reserve = !vcpu_get_flag(vcpu, IN_NESTED_ERET) &&
+ !vcpu_get_flag(vcpu, IN_NESTED_EXCEPTION);
+
+ __kvm_pmu_load(vcpu, reserve);
+}
+
+static void __kvm_pmu_put(struct kvm_vcpu *vcpu, bool release_counters)
{
struct arm_pmu *pmu = vcpu->kvm->arch.arm_pmu;
unsigned long guest_counters;
@@ -533,12 +603,13 @@ void kvm_pmu_put(struct kvm_vcpu *vcpu)
if (!kvm_pmu_is_partitioned(vcpu->kvm) || !vcpu->wants_to_run || !pmu)
return;
- local_irq_save(flags);
+ preempt_disable();
if (!cpumask_test_cpu(smp_processor_id(), &pmu->supported_cpus)) {
- local_irq_restore(flags);
+ preempt_enable();
return;
}
+ local_irq_save(flags);
guest_counters = kvm_vcpu_pmu_guest_counter_mask(vcpu);
mask = guest_counters;
@@ -595,5 +666,24 @@ void kvm_pmu_put(struct kvm_vcpu *vcpu)
__vcpu_rmw_sys_reg(vcpu, PMOVSSET_EL0, |=, val & mask);
write_sysreg(val & mask, pmovsclr_el0);
+ if (release_counters)
+ kvm_pmu_set_guest_counters(pmu, 0);
local_irq_restore(flags);
+ preempt_enable();
+}
+
+/**
+ * kvm_pmu_put() - Put untrapped PMU registers
+ * @vcpu: Pointer to struct kvm_vcpu
+ *
+ * Put all untrapped PMU registers from the PCPU into the VCPU. Mask
+ * to only bits belonging to guest-reserved counters and leave
+ * host-reserved counters alone in bitmask registers.
+ */
+void kvm_pmu_put(struct kvm_vcpu *vcpu)
+{
+ bool release = !vcpu_get_flag(vcpu, IN_NESTED_ERET) &&
+ !vcpu_get_flag(vcpu, IN_NESTED_EXCEPTION);
+
+ __kvm_pmu_put(vcpu, release);
}
diff --git a/drivers/perf/arm_pmu.c b/drivers/perf/arm_pmu.c
index b1851d1c594b1..2b5bbbd801af8 100644
--- a/drivers/perf/arm_pmu.c
+++ b/drivers/perf/arm_pmu.c
@@ -349,8 +349,15 @@ armpmu_add(struct perf_event *event, int flags)
/* If we don't have a space for the counter then finish early. */
idx = armpmu->get_event_idx(hw_events, event);
- if (idx < 0)
+ if (idx < 0) {
+ if (!bitmap_equal(hw_events->cntr_mask, armpmu->cntr_mask,
+ ARMPMU_MAX_HWEVENTS)) {
+ hw_events->host_squeezed = true;
+ if (event->group_leader->attr.pinned)
+ pr_warn_once("perf: Pinned host event squeezed out by KVM guest PMU partition\n");
+ }
return idx;
+ }
/* The newly-allocated counter should be empty */
WARN_ON_ONCE(hw_events->events[idx]);
diff --git a/include/linux/perf/arm_pmu.h b/include/linux/perf/arm_pmu.h
index be1e345e99a77..45658273ffa86 100644
--- a/include/linux/perf/arm_pmu.h
+++ b/include/linux/perf/arm_pmu.h
@@ -76,6 +76,7 @@ struct pmu_hw_events {
/* Active events requesting branch records */
unsigned int branch_users;
+ bool host_squeezed;
};
enum armpmu_attr_groups {
--
2.56.1.708.g77570a6a1d-goog
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v10 23/33] KVM: arm64: Implement lazy PMU context swaps
2026-11-19 18:51 [PATCH v10 00/33] ARM64 PMU Partitioning Colton Lewis
` (21 preceding siblings ...)
2026-11-19 18:52 ` [PATCH v10 22/33] KVM: arm64: Apply dynamic guest counter reservations Colton Lewis
@ 2026-11-19 18:52 ` Colton Lewis
2026-11-19 19:24 ` sashiko-bot
2026-11-19 18:52 ` [PATCH v10 24/33] perf: arm_pmuv3: Handle IRQs for Partitioned PMU guest counters Colton Lewis
` (9 subsequent siblings)
32 siblings, 1 reply; 67+ messages in thread
From: Colton Lewis @ 2026-11-19 18:52 UTC (permalink / raw)
To: Marc Zyngier, Oliver Upton
Cc: Catalin Marinas, Will Deacon, Mark Rutland, Peter Zijlstra,
Ingo Molnar, Paolo Bonzini, Russell King, Jonathan Corbet,
Fuad Tabba, Joey Gouly, Suzuki K Poulose, Zenghui Yu, Shuah Khan,
James Clark, Alexandru Elisei, Mingwei Zhang, Ganapatrao Kulkarni,
kvmarm, linux-arm-kernel, kvm, linux-perf-users, linux-kselftest,
linux-doc, linux-kernel
Since many guests never use the PMU, avoid reserving hardware counters
and context swapping PMU registers on every vCPU load and put until the
guest actively enables counting or overflow interrupts.
Track per-vCPU PMU ownership via enum vcpu_pmu_register_access
(VCPU_PMU_ACCESS_FREE vs. VCPU_PMU_ACCESS_GUEST_OWNED):
- While FREE, keep MDCR_EL2.HPMN at the full host counter count with TPM
and TPMCR set so host perf retains all counters including PMCCNTR_EL0.
Trapped reads and non-enabling writes (such as guest boot PMU probes)
operate on shadow sys_regs[] without claiming hardware counters.
- When the guest enables counting or interrupts (setting PMCR_EL0.E,
enabling MDCR_EL2.HPME with hypervisor counters, or setting guest bits
in PMCNTENSET_EL0 or PMINTENSET_EL1), transition to GUEST_OWNED via
kvm_pmu_set_guest_owned(), reserve the guest counter partition, and
load guest state into hardware.
Add kvm_pmu_reload_part() to service KVM_REQ_RELOAD_PMU for partitioned
VMs (triggered by MDCR_EL2 or PMUSERENR_EL0 trap changes, vCPU reset,
and KVM_SET_ONE_REG writes). Unlike the emulated PMU, which only masks
shadow registers and reprograms perf_events, a partitioned PMU loaded on
the CPU must flush live hardware registers to sys_regs[], mask
unimplemented counter bits, recompute FGT, MDCR_EL2, and hardware event
filters, and reload hardware without dropping its host counter
reservation. When not yet loaded (such as after vCPU reset or
KVM_SET_ONE_REG), kvm_pmu_reload_part() transitions to GUEST_OWNED only
if PMCR_EL0.E or MDCR_EL2.HPME is enabled, avoiding false activation
from reset_unknown() poison bits in PMCNTENSET_EL0 or PMINTENSET_EL1.
Signed-off-by: Colton Lewis <coltonlewis@google.com>
---
arch/arm64/include/asm/kvm_host.h | 1 +
arch/arm64/include/asm/kvm_types.h | 6 +-
arch/arm64/kvm/debug.c | 4 +-
arch/arm64/kvm/pmu-direct.c | 139 ++++++++++++++++++++++++++---
arch/arm64/kvm/pmu-emul.c | 3 +
arch/arm64/kvm/pmu.c | 5 ++
arch/arm64/kvm/sys_regs.c | 14 ++-
include/kvm/arm_pmu.h | 14 +++
8 files changed, 166 insertions(+), 20 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 103c25001c0d2..1fcb12995c34c 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -1501,6 +1501,7 @@ static inline bool kvm_system_needs_idmapped_vectors(void)
return cpus_have_final_cap(ARM64_SPECTRE_V3A);
}
+void kvm_arm_setup_mdcr_el2(struct kvm_vcpu *vcpu);
void kvm_init_host_debug_data(void);
void kvm_debug_init_vhe(void);
void kvm_vcpu_load_debug(struct kvm_vcpu *vcpu);
diff --git a/arch/arm64/include/asm/kvm_types.h b/arch/arm64/include/asm/kvm_types.h
index 9a126b9e2d7c9..4e39cbc80aa0b 100644
--- a/arch/arm64/include/asm/kvm_types.h
+++ b/arch/arm64/include/asm/kvm_types.h
@@ -4,5 +4,9 @@
#define KVM_ARCH_NR_OBJS_PER_MEMORY_CACHE 40
-#endif /* _ASM_ARM64_KVM_TYPES_H */
+enum vcpu_pmu_register_access {
+ VCPU_PMU_ACCESS_FREE,
+ VCPU_PMU_ACCESS_GUEST_OWNED,
+};
+#endif /* _ASM_ARM64_KVM_TYPES_H */
diff --git a/arch/arm64/kvm/debug.c b/arch/arm64/kvm/debug.c
index a1cb4c076bdb4..ec9f942b452d8 100644
--- a/arch/arm64/kvm/debug.c
+++ b/arch/arm64/kvm/debug.c
@@ -36,7 +36,7 @@ static int cpu_has_spe(u64 dfr0)
* - Self-hosted Trace Filter controls (MDCR_EL2_TTRF)
* - Self-hosted Trace (MDCR_EL2_TTRF/MDCR_EL2_E2TB)
*/
-static void kvm_arm_setup_mdcr_el2(struct kvm_vcpu *vcpu)
+void kvm_arm_setup_mdcr_el2(struct kvm_vcpu *vcpu)
{
unsigned long daif;
u64 mdcr;
@@ -55,7 +55,7 @@ static void kvm_arm_setup_mdcr_el2(struct kvm_vcpu *vcpu)
MDCR_EL2_TDRA |
MDCR_EL2_TDOSA);
- if (kvm_pmu_is_partitioned(vcpu->kvm)) {
+ if (kvm_pmu_is_partitioned(vcpu->kvm) && vcpu->arch.pmu.loaded_on_cpu) {
u8 nr_host_cntr = *host_data_ptr(nr_event_counters);
u8 nr_guest_cntr = vcpu->kvm->arch.nr_pmu_counters;
diff --git a/arch/arm64/kvm/pmu-direct.c b/arch/arm64/kvm/pmu-direct.c
index cc4ff00d6f41e..c2ea1ce5efe74 100644
--- a/arch/arm64/kvm/pmu-direct.c
+++ b/arch/arm64/kvm/pmu-direct.c
@@ -11,6 +11,7 @@
#include <linux/perf/arm_pmuv3.h>
#include <asm/arm_pmuv3.h>
+#include <asm/daifflags.h>
#include <asm/kvm_emulate.h>
#include <asm/virt.h>
@@ -68,13 +69,9 @@ bool kvm_pmu_is_partitioned(struct kvm *kvm)
static bool kvm_pmu_part_on_cpu(struct kvm_vcpu *vcpu)
{
- struct arm_pmu *pmu = vcpu->kvm->arch.arm_pmu;
-
lockdep_assert_preemption_disabled();
- return kvm_pmu_is_partitioned(vcpu->kvm) &&
- vcpu == kvm_get_running_vcpu() && pmu &&
- cpumask_test_cpu(smp_processor_id(), &pmu->supported_cpus);
+ return vcpu == kvm_get_running_vcpu() && vcpu->arch.pmu.loaded_on_cpu;
}
static u64 kvm_vcpu_pmu_guest_counter_mask(struct kvm_vcpu *vcpu)
@@ -92,6 +89,8 @@ static u64 kvm_vcpu_pmu_guest_counter_mask(struct kvm_vcpu *vcpu)
return 0;
}
+static void kvm_pmu_set_guest_owned(struct kvm_vcpu *vcpu);
+
static void kvm_pmu_direct_pmcr_write(struct kvm_vcpu *vcpu, u64 val)
{
bool reset_p = val & ARMV8_PMU_PMCR_P;
@@ -120,6 +119,9 @@ static void kvm_pmu_direct_pmcr_write(struct kvm_vcpu *vcpu, u64 val)
__vcpu_assign_sys_reg(vcpu, PMEVCNTR0_EL0 + i, 0);
}
+ if (val & ARMV8_PMU_PMCR_E)
+ kvm_pmu_set_guest_owned(vcpu);
+
if (!kvm_pmu_part_on_cpu(vcpu))
return;
@@ -230,7 +232,6 @@ static u64 kvm_pmu_direct_cntr_read(struct kvm_vcpu *vcpu, u8 idx)
return read_pmevcntrn(idx);
}
-
static void kvm_pmu_direct_cntr_write(struct kvm_vcpu *vcpu, u8 idx, u64 val)
{
lockdep_assert_preemption_disabled();
@@ -412,6 +413,8 @@ void kvm_pmu_direct_reg_write(struct kvm_vcpu *vcpu, unsigned int reg,
val &= kvm_vcpu_pmu_guest_counter_mask(vcpu);
if (set) {
__vcpu_rmw_sys_reg(vcpu, PMCNTENSET_EL0, |=, val);
+ if (val)
+ kvm_pmu_set_guest_owned(vcpu);
if (kvm_pmu_part_on_cpu(vcpu))
write_sysreg(val, pmcntenset_el0);
} else {
@@ -424,6 +427,8 @@ void kvm_pmu_direct_reg_write(struct kvm_vcpu *vcpu, unsigned int reg,
val &= kvm_vcpu_pmu_guest_counter_mask(vcpu);
if (set) {
__vcpu_rmw_sys_reg(vcpu, PMINTENSET_EL1, |=, val);
+ if (val)
+ kvm_pmu_set_guest_owned(vcpu);
if (kvm_pmu_part_on_cpu(vcpu))
write_sysreg(val, pmintenset_el1);
} else {
@@ -444,13 +449,18 @@ void kvm_pmu_direct_reg_write(struct kvm_vcpu *vcpu, unsigned int reg,
}
local_irq_restore(flags);
break;
- case PMUSERENR_EL0:
+ case PMUSERENR_EL0: {
+ u64 old = __vcpu_sys_reg(vcpu, reg);
+
val &= ARMV8_PMU_USERENR_MASK;
if (kvm_pmu_part_on_cpu(vcpu) &&
!(vcpu->arch.mdcr_el2 & MDCR_EL2_TPM))
write_sysreg(val, pmuserenr_el0);
__vcpu_assign_sys_reg(vcpu, reg, val);
+ if (vcpu_has_nv(vcpu) && ((old ^ val) & ARMV8_PMU_USERENR_MASK))
+ kvm_make_request(KVM_REQ_RELOAD_PMU, vcpu);
break;
+ }
default:
WARN_ON(1);
break;
@@ -468,7 +478,12 @@ void kvm_pmu_direct_reg_write(struct kvm_vcpu *vcpu, unsigned int reg,
*/
u64 kvm_pmu_guest_counter_mask(void)
{
- return kvm_vcpu_pmu_guest_counter_mask(kvm_get_running_vcpu());
+ struct kvm_vcpu *vcpu = kvm_get_running_vcpu();
+
+ if (vcpu && vcpu->arch.pmu.loaded_on_cpu)
+ return kvm_vcpu_pmu_guest_counter_mask(vcpu);
+
+ return 0;
}
static void kvm_pmu_apply_event_filter(struct kvm_vcpu *vcpu)
@@ -491,6 +506,7 @@ static void __kvm_pmu_load(struct kvm_vcpu *vcpu, bool reserve_counters)
struct arm_pmu *pmu = vcpu->kvm->arch.arm_pmu;
unsigned long guest_counters;
unsigned long flags;
+ unsigned long daif;
u64 mask;
u8 i;
u64 val;
@@ -499,7 +515,9 @@ static void __kvm_pmu_load(struct kvm_vcpu *vcpu, bool reserve_counters)
* If we aren't guest-owned then we know the guest isn't using
* the PMU anyway, so no need to bother with the swap.
*/
- if (!kvm_pmu_is_partitioned(vcpu->kvm) || !vcpu->wants_to_run || !pmu)
+ if (!kvm_pmu_is_partitioned(vcpu->kvm) ||
+ kvm_pmu_get_access(vcpu) != VCPU_PMU_ACCESS_GUEST_OWNED ||
+ vcpu->arch.pmu.loaded_on_cpu || !vcpu->wants_to_run || !pmu)
return;
preempt_disable();
@@ -514,6 +532,22 @@ static void __kvm_pmu_load(struct kvm_vcpu *vcpu, bool reserve_counters)
if (reserve_counters)
kvm_pmu_set_guest_counters(pmu, guest_counters);
+ daif = local_daif_save();
+
+ /*
+ * Ensure guest counters, interrupts, and overflow flags are quiesced
+ * in hardware before lowering MDCR_EL2.HPMN and marking the partition
+ * live on this CPU.
+ */
+ write_sysreg(guest_counters, pmcntenclr_el0);
+ write_sysreg(guest_counters, pmintenclr_el1);
+ write_sysreg(guest_counters, pmovsclr_el0);
+ isb();
+
+ vcpu->arch.pmu.loaded_on_cpu = true;
+ kvm_arm_setup_mdcr_el2(vcpu);
+ isb();
+
kvm_pmu_apply_event_filter(vcpu);
for_each_set_bit(i, &guest_counters, ARMPMU_MAX_HWEVENTS) {
@@ -565,6 +599,7 @@ static void __kvm_pmu_load(struct kvm_vcpu *vcpu, bool reserve_counters)
write_sysreg(val & mask, pmintenset_el1);
write_sysreg(~val & mask, pmintenclr_el1);
+ local_daif_restore(daif);
local_irq_restore(flags);
preempt_enable();
}
@@ -582,6 +617,9 @@ void kvm_pmu_load(struct kvm_vcpu *vcpu)
bool reserve = !vcpu_get_flag(vcpu, IN_NESTED_ERET) &&
!vcpu_get_flag(vcpu, IN_NESTED_EXCEPTION);
+ if (reserve && kvm_test_request(KVM_REQ_RELOAD_PMU, vcpu))
+ return;
+
__kvm_pmu_load(vcpu, reserve);
}
@@ -590,29 +628,34 @@ static void __kvm_pmu_put(struct kvm_vcpu *vcpu, bool release_counters)
struct arm_pmu *pmu = vcpu->kvm->arch.arm_pmu;
unsigned long guest_counters;
unsigned long flags;
+ unsigned long daif;
u64 pmcr_mask;
u64 mask;
u8 i;
u64 val;
/*
- * If we aren't guest-owned then we know the guest is not
+ * If we aren't loaded on the CPU then we know the guest is not
* accessing the PMU anyway, so no need to bother with the
* swap.
*/
- if (!kvm_pmu_is_partitioned(vcpu->kvm) || !vcpu->wants_to_run || !pmu)
+ if (!kvm_pmu_is_partitioned(vcpu->kvm) ||
+ !vcpu->arch.pmu.loaded_on_cpu || !pmu)
return;
preempt_disable();
+ local_irq_save(flags);
if (!cpumask_test_cpu(smp_processor_id(), &pmu->supported_cpus)) {
+ local_irq_restore(flags);
preempt_enable();
return;
}
- local_irq_save(flags);
guest_counters = kvm_vcpu_pmu_guest_counter_mask(vcpu);
mask = guest_counters;
+ daif = local_daif_save();
+
/* Mask these to only save the guest relevant bits. */
val = read_sysreg(pmcntenset_el0);
__vcpu_assign_sys_reg(vcpu, PMCNTENSET_EL0, val & mask);
@@ -657,7 +700,7 @@ static void __kvm_pmu_put(struct kvm_vcpu *vcpu, bool release_counters)
val = ARMV8_PMU_PMCR_LC;
if (pmu->pmuver >= ID_AA64DFR0_EL1_PMUVer_V3P5)
val |= ARMV8_PMU_PMCR_LP;
- if (vcpu->arch.mdcr_el2 & MDCR_EL2_HPME)
+ if (READ_ONCE(vcpu->arch.mdcr_el2) & MDCR_EL2_HPME)
val |= ARMV8_PMU_PMCR_E;
write_sysreg(val, pmcr_el0);
@@ -666,6 +709,12 @@ static void __kvm_pmu_put(struct kvm_vcpu *vcpu, bool release_counters)
__vcpu_rmw_sys_reg(vcpu, PMOVSSET_EL0, |=, val & mask);
write_sysreg(val & mask, pmovsclr_el0);
+ vcpu->arch.pmu.loaded_on_cpu = false;
+ kvm_arm_setup_mdcr_el2(vcpu);
+ isb();
+
+ local_daif_restore(daif);
+
if (release_counters)
kvm_pmu_set_guest_counters(pmu, 0);
local_irq_restore(flags);
@@ -687,3 +736,67 @@ void kvm_pmu_put(struct kvm_vcpu *vcpu)
__kvm_pmu_put(vcpu, release);
}
+
+/**
+ * kvm_pmu_reload_part() - Reload partitioned PMU traps, filters, and state
+ * @vcpu: Pointer to struct kvm_vcpu
+ *
+ * Refresh MDCR_EL2 trap bits, event filters, and hardware register state
+ * without releasing the host counter reservation when the partitioned PMU is
+ * already loaded on the CPU.
+ */
+void kvm_pmu_reload_part(struct kvm_vcpu *vcpu)
+{
+ u64 mask = kvm_pmu_implemented_counter_mask(vcpu);
+ bool was_loaded;
+
+ guard(preempt)();
+
+ was_loaded = vcpu->arch.pmu.loaded_on_cpu;
+ if (was_loaded)
+ __kvm_pmu_put(vcpu, false);
+
+ if (!kvm_has_feat(vcpu->kvm, ID_AA64DFR0_EL1, PMUVer, V3P5))
+ __vcpu_rmw_sys_reg(vcpu, PMCR_EL0, &=, ~ARMV8_PMU_PMCR_LP);
+
+ __vcpu_rmw_sys_reg(vcpu, PMOVSSET_EL0, &=, mask);
+ __vcpu_rmw_sys_reg(vcpu, PMINTENSET_EL1, &=, mask);
+ __vcpu_rmw_sys_reg(vcpu, PMCNTENSET_EL0, &=, mask);
+
+ kvm_vcpu_load_fgt(vcpu);
+ if (vcpu == kvm_get_running_vcpu() && has_vhe() &&
+ cpus_have_final_cap(ARM64_HAS_FGT)) {
+ write_sysreg_s(*vcpu_fgt(vcpu, HDFGRTR_EL2), SYS_HDFGRTR_EL2);
+ write_sysreg_s(*vcpu_fgt(vcpu, HDFGWTR_EL2), SYS_HDFGWTR_EL2);
+ isb();
+ }
+
+ if (was_loaded) {
+ __kvm_pmu_load(vcpu, false);
+ } else {
+ bool hpme_active = (__vcpu_sys_reg(vcpu, MDCR_EL2) &
+ MDCR_EL2_HPME) &&
+ kvm_pmu_hyp_counter_mask(vcpu);
+
+ if ((__vcpu_sys_reg(vcpu, PMCR_EL0) & ARMV8_PMU_PMCR_E) ||
+ hpme_active)
+ kvm_pmu_set_guest_owned(vcpu);
+ else if (kvm_pmu_get_access(vcpu) == VCPU_PMU_ACCESS_GUEST_OWNED)
+ __kvm_pmu_load(vcpu, true);
+
+ if (!vcpu->arch.pmu.loaded_on_cpu &&
+ vcpu == kvm_get_running_vcpu())
+ kvm_arm_setup_mdcr_el2(vcpu);
+ }
+}
+
+static void kvm_pmu_set_guest_owned(struct kvm_vcpu *vcpu)
+{
+ lockdep_assert_preemption_disabled();
+
+ if (kvm_pmu_is_partitioned(vcpu->kvm) &&
+ vcpu == kvm_get_running_vcpu()) {
+ vcpu->arch.pmu.access = VCPU_PMU_ACCESS_GUEST_OWNED;
+ __kvm_pmu_load(vcpu, true);
+ }
+}
diff --git a/arch/arm64/kvm/pmu-emul.c b/arch/arm64/kvm/pmu-emul.c
index 9eff208547746..da66162c68042 100644
--- a/arch/arm64/kvm/pmu-emul.c
+++ b/arch/arm64/kvm/pmu-emul.c
@@ -625,6 +625,9 @@ void kvm_pmu_nested_transition(struct kvm_vcpu *vcpu)
unsigned long mask;
int i;
+ if (kvm_pmu_is_partitioned(vcpu->kvm))
+ return;
+
mask = __vcpu_sys_reg(vcpu, PMCNTENSET_EL0);
for_each_set_bit(i, &mask, 32) {
struct kvm_pmc *pmc = kvm_vcpu_idx_to_pmc(vcpu, i);
diff --git a/arch/arm64/kvm/pmu.c b/arch/arm64/kvm/pmu.c
index d3fd438a57b09..7389d61957fa9 100644
--- a/arch/arm64/kvm/pmu.c
+++ b/arch/arm64/kvm/pmu.c
@@ -925,6 +925,11 @@ void kvm_vcpu_reload_pmu(struct kvm_vcpu *vcpu)
{
u64 mask = kvm_pmu_implemented_counter_mask(vcpu);
+ if (kvm_pmu_is_partitioned(vcpu->kvm)) {
+ kvm_pmu_reload_part(vcpu);
+ return;
+ }
+
__vcpu_rmw_sys_reg(vcpu, PMOVSSET_EL0, &=, mask);
__vcpu_rmw_sys_reg(vcpu, PMINTENSET_EL1, &=, mask);
__vcpu_rmw_sys_reg(vcpu, PMCNTENSET_EL0, &=, mask);
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index db6f3adcaae7c..6706f57aef8d5 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -3181,10 +3181,11 @@ static bool access_mdcr(struct kvm_vcpu *vcpu,
__vcpu_assign_sys_reg(vcpu, MDCR_EL2, val);
/*
- * Request a reload of the PMU to enable/disable the counters
- * affected by HPME.
+ * Request a reload of the PMU to update the counters and filters
+ * affected by MDCR_EL2.
*/
- if ((old ^ val) & MDCR_EL2_HPME)
+ if ((old ^ val) & (MDCR_EL2_HPME | MDCR_EL2_HPMN | MDCR_EL2_HPMD |
+ MDCR_EL2_HCCD | MDCR_EL2_TPM | MDCR_EL2_TPMCR))
kvm_make_request(KVM_REQ_RELOAD_PMU, vcpu);
return true;
@@ -5455,8 +5456,10 @@ void kvm_reset_sys_regs(struct kvm_vcpu *vcpu)
set_bit(KVM_ARCH_FLAG_ID_REGS_INITIALIZED, &kvm->arch.flags);
- if (kvm_vcpu_has_pmu(vcpu))
+ if (kvm_vcpu_has_pmu(vcpu)) {
+ kvm_pmu_reset_access(vcpu);
kvm_make_request(KVM_REQ_RELOAD_PMU, vcpu);
+ }
}
/**
@@ -5690,6 +5693,9 @@ int kvm_sys_reg_set_user(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg,
ret = 0;
}
+ if (!ret && kvm_vcpu_has_pmu(vcpu) && kvm_pmu_is_partitioned(vcpu->kvm))
+ kvm_make_request(KVM_REQ_RELOAD_PMU, vcpu);
+
return ret;
}
diff --git a/include/kvm/arm_pmu.h b/include/kvm/arm_pmu.h
index 4580f62234128..e3f5b9a1a4817 100644
--- a/include/kvm/arm_pmu.h
+++ b/include/kvm/arm_pmu.h
@@ -7,6 +7,7 @@
#ifndef __ASM_ARM_KVM_PMU_H
#define __ASM_ARM_KVM_PMU_H
+#include <linux/kvm_types.h>
#include <linux/perf_event.h>
#include <linux/perf/arm_pmuv3.h>
#include <linux/perf/arm_pmu.h>
@@ -43,6 +44,8 @@ struct kvm_pmu {
struct kvm_pmc pmc[KVM_ARMV8_PMU_MAX_COUNTERS];
int irq_num;
bool created;
+ bool loaded_on_cpu;
+ enum vcpu_pmu_register_access access;
};
struct arm_pmu_entry {
@@ -101,10 +104,14 @@ bool kvm_pmu_is_partitioned(struct kvm *kvm);
u64 kvm_pmu_direct_reg_read(struct kvm_vcpu *vcpu, unsigned int reg);
void kvm_pmu_direct_reg_write(struct kvm_vcpu *vcpu, unsigned int reg,
u64 val, bool set);
+void kvm_pmu_reload_part(struct kvm_vcpu *vcpu);
u64 kvm_pmu_guest_counter_mask(void);
void kvm_pmu_load(struct kvm_vcpu *vcpu);
void kvm_pmu_put(struct kvm_vcpu *vcpu);
+#define kvm_pmu_get_access(vcpu) ((vcpu)->arch.pmu.access)
+#define kvm_pmu_reset_access(vcpu) ((vcpu)->arch.pmu.access = VCPU_PMU_ACCESS_FREE)
+
/*
* Updates the vcpu's view of the pmu events for this cpu.
* Must be called before every vcpu run after disabling interrupts, to ensure
@@ -126,6 +133,7 @@ bool kvm_pmu_counter_is_hyp(struct kvm_vcpu *vcpu, unsigned int idx);
void kvm_pmu_nested_transition(struct kvm_vcpu *vcpu);
#else
struct kvm_pmu {
+ bool loaded_on_cpu;
};
static inline bool kvm_supports_guest_pmuv3(void)
@@ -144,6 +152,11 @@ static inline bool kvm_pmu_is_partitioned(struct kvm *kvm)
return false;
}
+static inline enum vcpu_pmu_register_access kvm_pmu_get_access(struct kvm_vcpu *vcpu)
+{
+ return VCPU_PMU_ACCESS_FREE;
+}
+static inline void kvm_pmu_reset_access(struct kvm_vcpu *vcpu) {}
static inline u64 kvm_pmu_direct_reg_read(struct kvm_vcpu *vcpu,
unsigned int reg)
{
@@ -152,6 +165,7 @@ static inline u64 kvm_pmu_direct_reg_read(struct kvm_vcpu *vcpu,
static inline void kvm_pmu_direct_reg_write(struct kvm_vcpu *vcpu,
unsigned int reg, u64 val,
bool set) {}
+static inline void kvm_pmu_reload_part(struct kvm_vcpu *vcpu) {}
static inline void kvm_pmu_load(struct kvm_vcpu *vcpu) {}
static inline void kvm_pmu_put(struct kvm_vcpu *vcpu) {}
static inline void kvm_pmu_set_counter_value(struct kvm_vcpu *vcpu,
--
2.56.1.708.g77570a6a1d-goog
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v10 24/33] perf: arm_pmuv3: Handle IRQs for Partitioned PMU guest counters
2026-11-19 18:51 [PATCH v10 00/33] ARM64 PMU Partitioning Colton Lewis
` (22 preceding siblings ...)
2026-11-19 18:52 ` [PATCH v10 23/33] KVM: arm64: Implement lazy PMU context swaps Colton Lewis
@ 2026-11-19 18:52 ` Colton Lewis
2026-11-19 19:22 ` sashiko-bot
2026-11-19 18:52 ` [PATCH v10 25/33] KVM: arm64: Detect overflows for the Partitioned PMU Colton Lewis
` (8 subsequent siblings)
32 siblings, 1 reply; 67+ messages in thread
From: Colton Lewis @ 2026-11-19 18:52 UTC (permalink / raw)
To: Marc Zyngier, Oliver Upton
Cc: Catalin Marinas, Will Deacon, Mark Rutland, Peter Zijlstra,
Ingo Molnar, Paolo Bonzini, Russell King, Jonathan Corbet,
Fuad Tabba, Joey Gouly, Suzuki K Poulose, Zenghui Yu, Shuah Khan,
James Clark, Alexandru Elisei, Mingwei Zhang, Ganapatrao Kulkarni,
kvmarm, linux-arm-kernel, kvm, linux-perf-users, linux-kselftest,
linux-doc, linux-kernel
Because ARM hardware does not directly inject PMU PPIs into guests,
overflows on partitioned guest counters raise physical PMU interrupts
that enter the host's armv8pmu_handle_irq() handler.
Update armv8pmu_handle_irq() to read PMOVSCLR_EL0 without blindly
clearing guest overflow bits:
- Pass pmovsr to kvm_pmu_handle_guest_irq(), which clears guest-reserved
overflow bits in hardware and latches them into the running vCPU's
shadow PMOVSSET_EL0 register.
- Mask out guest-reserved counter bits before clearing and processing
host perf overflows, returning IRQ_HANDLED early when only guest
counters overflowed or no host events are active.
Also add kvm_pmu_host_start() and kvm_pmu_host_stop() so
armv8pmu_start() and armv8pmu_stop() toggle MDCR_EL2.HPME rather than
PMCR_EL0.E while a partitioned guest's PMU state is loaded on the CPU
(leaving guest counters 0..HPMN-1 and PMCCNTR_EL0 under guest PMCR_EL0.E
control), guarding local_daif_save() with !in_nmi() for GICv3 pseudo-NMI
interrupt handlers.
Signed-off-by: Colton Lewis <coltonlewis@google.com>
---
arch/arm/include/asm/arm_pmuv3.h | 8 +++
arch/arm64/include/asm/arm_pmuv3.h | 5 ++
arch/arm64/kvm/pmu-direct.c | 102 +++++++++++++++++++++++++++++
drivers/perf/arm_pmuv3.c | 34 +++++-----
include/kvm/arm_pmu.h | 7 ++
5 files changed, 141 insertions(+), 15 deletions(-)
diff --git a/arch/arm/include/asm/arm_pmuv3.h b/arch/arm/include/asm/arm_pmuv3.h
index 71cf03ab58cbb..d8051ea261481 100644
--- a/arch/arm/include/asm/arm_pmuv3.h
+++ b/arch/arm/include/asm/arm_pmuv3.h
@@ -180,6 +180,11 @@ static inline void write_pmintenset(u32 val)
write_sysreg(val, PMINTENSET);
}
+static inline u32 read_pmintenset(void)
+{
+ return read_sysreg(PMINTENSET);
+}
+
static inline void write_pmintenclr(u32 val)
{
write_sysreg(val, PMINTENCLR);
@@ -231,6 +236,9 @@ static inline u64 kvm_pmu_guest_counter_mask(void)
{
return 0;
}
+static inline void kvm_pmu_handle_guest_irq(u64 pmovsr) {}
+static inline bool kvm_pmu_host_start(void) { return false; }
+static inline bool kvm_pmu_host_stop(void) { return false; }
/* PMU Version in DFR Register */
#define ARMV8_PMU_DFR_VER_NI 0
diff --git a/arch/arm64/include/asm/arm_pmuv3.h b/arch/arm64/include/asm/arm_pmuv3.h
index cf2b2212e00a2..9037a28365951 100644
--- a/arch/arm64/include/asm/arm_pmuv3.h
+++ b/arch/arm64/include/asm/arm_pmuv3.h
@@ -110,6 +110,11 @@ static inline void write_pmintenset(u64 val)
write_sysreg(val, pmintenset_el1);
}
+static inline u64 read_pmintenset(void)
+{
+ return read_sysreg(pmintenset_el1);
+}
+
static inline void write_pmintenclr(u64 val)
{
write_sysreg(val, pmintenclr_el1);
diff --git a/arch/arm64/kvm/pmu-direct.c b/arch/arm64/kvm/pmu-direct.c
index c2ea1ce5efe74..21357decce214 100644
--- a/arch/arm64/kvm/pmu-direct.c
+++ b/arch/arm64/kvm/pmu-direct.c
@@ -800,3 +800,105 @@ static void kvm_pmu_set_guest_owned(struct kvm_vcpu *vcpu)
__kvm_pmu_load(vcpu, true);
}
}
+
+/**
+ * kvm_pmu_handle_guest_irq() - Record IRQs in guest counters
+ * @pmovsr: Overflow flags reported by driver
+ *
+ * Set overflow flags in guest-reserved counters in the VCPU register
+ * for the guest to clear later.
+ */
+void kvm_pmu_handle_guest_irq(u64 pmovsr)
+{
+ struct kvm_vcpu *vcpu = kvm_get_running_vcpu();
+ u64 mask = kvm_pmu_guest_counter_mask();
+ u64 govf = pmovsr & mask;
+
+ if (!govf)
+ return;
+
+ write_pmovsclr(govf);
+
+ if (!vcpu || !vcpu->arch.pmu.loaded_on_cpu)
+ return;
+
+ __vcpu_rmw_sys_reg(vcpu, PMOVSSET_EL0, |=, govf);
+}
+
+/**
+ * kvm_pmu_host_start() - Enable host PMU counters while guest owns PMU
+ *
+ * When a partitioned guest owns the PMU, host counters (HPMN..N-1) are
+ * gated by MDCR_EL2.HPME rather than PMCR_EL0.E.
+ *
+ * Return: True if handled via MDCR_EL2.HPME, false if caller should use PMCR_EL0.E
+ */
+bool kvm_pmu_host_start(void)
+{
+ struct kvm_vcpu *vcpu = kvm_get_running_vcpu();
+ unsigned long daif = 0;
+ bool nmi;
+ u64 mdcr;
+
+ if (!vcpu || !kvm_pmu_is_partitioned(vcpu->kvm) ||
+ !vcpu->arch.pmu.loaded_on_cpu)
+ return false;
+
+ nmi = in_nmi();
+ if (!nmi)
+ daif = local_daif_save();
+
+ if (!vcpu->arch.pmu.loaded_on_cpu) {
+ if (!nmi)
+ local_daif_restore(daif);
+ return false;
+ }
+
+ mdcr = READ_ONCE(vcpu->arch.mdcr_el2) | MDCR_EL2_HPME;
+ WRITE_ONCE(vcpu->arch.mdcr_el2, mdcr);
+ if (has_vhe())
+ write_sysreg(mdcr, mdcr_el2);
+ isb();
+ if (!nmi)
+ local_daif_restore(daif);
+ return true;
+}
+
+/**
+ * kvm_pmu_host_stop() - Disable host PMU counters while guest owns PMU
+ *
+ * When a partitioned guest owns the PMU, host counters (HPMN..N-1) are
+ * gated by MDCR_EL2.HPME rather than PMCR_EL0.E.
+ *
+ * Return: True if handled via MDCR_EL2.HPME, false if caller should use PMCR_EL0.E
+ */
+bool kvm_pmu_host_stop(void)
+{
+ struct kvm_vcpu *vcpu = kvm_get_running_vcpu();
+ unsigned long daif = 0;
+ bool nmi;
+ u64 mdcr;
+
+ if (!vcpu || !kvm_pmu_is_partitioned(vcpu->kvm) ||
+ !vcpu->arch.pmu.loaded_on_cpu)
+ return false;
+
+ nmi = in_nmi();
+ if (!nmi)
+ daif = local_daif_save();
+
+ if (!vcpu->arch.pmu.loaded_on_cpu) {
+ if (!nmi)
+ local_daif_restore(daif);
+ return false;
+ }
+
+ mdcr = READ_ONCE(vcpu->arch.mdcr_el2) & ~MDCR_EL2_HPME;
+ WRITE_ONCE(vcpu->arch.mdcr_el2, mdcr);
+ if (has_vhe())
+ write_sysreg(mdcr, mdcr_el2);
+ isb();
+ if (!nmi)
+ local_daif_restore(daif);
+ return true;
+}
diff --git a/drivers/perf/arm_pmuv3.c b/drivers/perf/arm_pmuv3.c
index 1676f13cc7c38..440f77cf0a2fc 100644
--- a/drivers/perf/arm_pmuv3.c
+++ b/drivers/perf/arm_pmuv3.c
@@ -777,18 +777,9 @@ static void armv8pmu_disable_event_irq(struct perf_event *event)
armv8pmu_disable_intens(BIT(event->hw.idx));
}
-static u64 armv8pmu_getreset_flags(void)
+static u64 armv8pmu_getovf_flags(void)
{
- u64 value;
-
- /* Read */
- value = read_pmovsclr();
-
- /* Write to clear flags */
- value &= ARMV8_PMU_CNT_MASK_ALL;
- write_pmovsclr(value);
-
- return value;
+ return read_pmovsclr() & ARMV8_PMU_CNT_MASK_ALL;
}
static void update_pmuserenr(u64 val)
@@ -878,7 +869,8 @@ static void armv8pmu_start(struct arm_pmu *cpu_pmu)
brbe_enable(cpu_pmu);
/* Enable all counters */
- armv8pmu_pmcr_write(armv8pmu_pmcr_read() | ARMV8_PMU_PMCR_E);
+ if (!kvm_pmu_host_start())
+ armv8pmu_pmcr_write(armv8pmu_pmcr_read() | ARMV8_PMU_PMCR_E);
}
static void armv8pmu_stop(struct arm_pmu *cpu_pmu)
@@ -889,7 +881,8 @@ static void armv8pmu_stop(struct arm_pmu *cpu_pmu)
brbe_disable();
/* Disable all counters */
- armv8pmu_pmcr_write(armv8pmu_pmcr_read() & ~ARMV8_PMU_PMCR_E);
+ if (!kvm_pmu_host_stop())
+ armv8pmu_pmcr_write(armv8pmu_pmcr_read() & ~ARMV8_PMU_PMCR_E);
}
static void read_branch_records(struct pmu_hw_events *cpuc,
@@ -911,9 +904,9 @@ static irqreturn_t armv8pmu_handle_irq(struct arm_pmu *cpu_pmu)
int idx;
/*
- * Get and reset the IRQ flags
+ * Get the IRQ flags
*/
- pmovsr = armv8pmu_getreset_flags();
+ pmovsr = armv8pmu_getovf_flags();
/*
* Did an overflow occur?
@@ -921,6 +914,17 @@ static irqreturn_t armv8pmu_handle_irq(struct arm_pmu *cpu_pmu)
if (!armv8pmu_has_overflowed(pmovsr))
return IRQ_NONE;
+ kvm_pmu_handle_guest_irq(pmovsr);
+
+ pmovsr &= ~kvm_pmu_guest_counter_mask();
+ if (!pmovsr)
+ return IRQ_HANDLED;
+
+ write_pmovsclr(pmovsr);
+
+ if (bitmap_empty(cpuc->used_mask, ARMPMU_MAX_HWEVENTS))
+ return IRQ_HANDLED;
+
/*
* Handle the counter(s) overflow(s)
*/
diff --git a/include/kvm/arm_pmu.h b/include/kvm/arm_pmu.h
index e3f5b9a1a4817..9800514019630 100644
--- a/include/kvm/arm_pmu.h
+++ b/include/kvm/arm_pmu.h
@@ -108,6 +108,9 @@ void kvm_pmu_reload_part(struct kvm_vcpu *vcpu);
u64 kvm_pmu_guest_counter_mask(void);
void kvm_pmu_load(struct kvm_vcpu *vcpu);
void kvm_pmu_put(struct kvm_vcpu *vcpu);
+void kvm_pmu_handle_guest_irq(u64 pmovsr);
+bool kvm_pmu_host_start(void);
+bool kvm_pmu_host_stop(void);
#define kvm_pmu_get_access(vcpu) ((vcpu)->arch.pmu.access)
#define kvm_pmu_reset_access(vcpu) ((vcpu)->arch.pmu.access = VCPU_PMU_ACCESS_FREE)
@@ -277,6 +280,10 @@ static inline u64 kvm_pmu_guest_counter_mask(void)
return 0;
}
+static inline void kvm_pmu_handle_guest_irq(u64 pmovsr) {}
+static inline bool kvm_pmu_host_start(void) { return false; }
+static inline bool kvm_pmu_host_stop(void) { return false; }
+
static inline bool has_kvm_pmu_partition_support(void)
{
return false;
--
2.56.1.708.g77570a6a1d-goog
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v10 25/33] KVM: arm64: Detect overflows for the Partitioned PMU
2026-11-19 18:51 [PATCH v10 00/33] ARM64 PMU Partitioning Colton Lewis
` (23 preceding siblings ...)
2026-11-19 18:52 ` [PATCH v10 24/33] perf: arm_pmuv3: Handle IRQs for Partitioned PMU guest counters Colton Lewis
@ 2026-11-19 18:52 ` Colton Lewis
2026-11-19 19:15 ` sashiko-bot
2026-11-19 18:52 ` [PATCH v10 26/33] KVM: arm64: Multiplex PMCR_EL0.E and MDCR_EL2.HPME for sub-partitioned L1 Colton Lewis
` (7 subsequent siblings)
32 siblings, 1 reply; 67+ messages in thread
From: Colton Lewis @ 2026-11-19 18:52 UTC (permalink / raw)
To: Marc Zyngier, Oliver Upton
Cc: Catalin Marinas, Will Deacon, Mark Rutland, Peter Zijlstra,
Ingo Molnar, Paolo Bonzini, Russell King, Jonathan Corbet,
Fuad Tabba, Joey Gouly, Suzuki K Poulose, Zenghui Yu, Shuah Khan,
James Clark, Alexandru Elisei, Mingwei Zhang, Ganapatrao Kulkarni,
kvmarm, linux-arm-kernel, kvm, linux-perf-users, linux-kselftest,
linux-doc, linux-kernel
Evaluate whether any enabled and unmasked guest counter has overflowed
in kvm_pmu_part_overflow_status() so KVM updates the virtual PMU
interrupt level in the vGIC (or userspace IRQ state) across VM entries
and host PMU overflow interrupts:
- Harvest any live guest overflow bits from physical pmovsset_el0 into
shadow PMOVSSET_EL0 and clear them in hardware when the partitioned
PMU is loaded on the local CPU.
- Check (PMOVSSET_EL0 & PMINTENSET_EL1) against PMCR_EL0.E for guest
counters 0..HPMN-1 and PMCCNTR_EL0, and against MDCR_EL2.HPME for
nested hypervisor counters HPMN..N-1.
- In kvm_pmu_handle_guest_irq(), raise KVM_REQ_IRQ_PENDING and kick the
vCPU (or queue overflow_work when running in NMI context) when an
unmasked guest overflow occurs.
Signed-off-by: Colton Lewis <coltonlewis@google.com>
---
arch/arm64/kvm/pmu-direct.c | 65 +++++++++++++++++++++++++++++++++++++
arch/arm64/kvm/pmu-emul.c | 4 +--
arch/arm64/kvm/pmu.c | 8 +++++
include/kvm/arm_pmu.h | 2 ++
4 files changed, 77 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/kvm/pmu-direct.c b/arch/arm64/kvm/pmu-direct.c
index 21357decce214..b0491d32b5d7b 100644
--- a/arch/arm64/kvm/pmu-direct.c
+++ b/arch/arm64/kvm/pmu-direct.c
@@ -823,6 +823,71 @@ void kvm_pmu_handle_guest_irq(u64 pmovsr)
return;
__vcpu_rmw_sys_reg(vcpu, PMOVSSET_EL0, |=, govf);
+
+ if (kvm_pmu_part_overflow_status(vcpu)) {
+ kvm_make_request(KVM_REQ_IRQ_PENDING, vcpu);
+
+ if (!in_nmi())
+ kvm_vcpu_kick(vcpu);
+ else
+ irq_work_queue(&vcpu->arch.pmu.overflow_work);
+ }
+}
+
+/**
+ * kvm_pmu_part_overflow_status() - Determine if any guest counters have overflowed
+ * @vcpu: Pointer to struct kvm_vcpu
+ *
+ * Determine if any guest counters have overflowed and therefore an
+ * IRQ needs to be injected into the guest. If an overflow is detected
+ * while access is VCPU_PMU_ACCESS_FREE (for example, after state
+ * restoration via KVM_SET_ONE_REG), transition the vCPU to
+ * VCPU_PMU_ACCESS_GUEST_OWNED.
+ *
+ * Return: True if there was an overflow, false otherwise
+ */
+bool kvm_pmu_part_overflow_status(struct kvm_vcpu *vcpu)
+{
+ u64 mask = kvm_vcpu_pmu_guest_counter_mask(vcpu);
+ u64 hyp_mask = kvm_pmu_hyp_counter_mask(vcpu);
+ u64 pmovs, pmint, pmcr, reg;
+ unsigned long flags;
+ bool overflow;
+
+ local_irq_save(flags);
+ if (vcpu->arch.pmu.loaded_on_cpu &&
+ vcpu == kvm_get_running_vcpu()) {
+ u64 hw_ovf = read_sysreg(pmovsset_el0) & mask;
+
+ if (hw_ovf) {
+ __vcpu_rmw_sys_reg(vcpu, PMOVSSET_EL0, |=, hw_ovf);
+ write_sysreg(hw_ovf, pmovsclr_el0);
+ }
+ pmint = read_pmintenset();
+ pmcr = read_pmcr();
+ } else {
+ pmint = __vcpu_sys_reg(vcpu, PMINTENSET_EL1);
+ pmcr = __vcpu_sys_reg(vcpu, PMCR_EL0);
+ }
+
+ pmovs = __vcpu_sys_reg(vcpu, PMOVSSET_EL0);
+ local_irq_restore(flags);
+
+ reg = pmovs & pmint & mask;
+ if (!(pmcr & ARMV8_PMU_PMCR_E))
+ reg &= hyp_mask;
+ if (hyp_mask && !(__vcpu_sys_reg(vcpu, MDCR_EL2) & MDCR_EL2_HPME))
+ reg &= ~hyp_mask;
+
+ overflow = !!reg;
+
+ if (overflow && !in_interrupt() &&
+ kvm_pmu_get_access(vcpu) == VCPU_PMU_ACCESS_FREE) {
+ scoped_guard(preempt)
+ kvm_pmu_set_guest_owned(vcpu);
+ }
+
+ return overflow;
}
/**
diff --git a/arch/arm64/kvm/pmu-emul.c b/arch/arm64/kvm/pmu-emul.c
index da66162c68042..a72bdc5a0b3ea 100644
--- a/arch/arm64/kvm/pmu-emul.c
+++ b/arch/arm64/kvm/pmu-emul.c
@@ -268,7 +268,7 @@ void kvm_pmu_reprogram_counter_mask(struct kvm_vcpu *vcpu, u64 val)
* counter where the values of the global enable control, PMOVSSET_EL0[n], and
* PMINTENSET_EL1[n] are all 1.
*/
-bool kvm_pmu_overflow_status(struct kvm_vcpu *vcpu)
+bool kvm_pmu_emul_overflow_status(struct kvm_vcpu *vcpu)
{
u64 reg = __vcpu_sys_reg(vcpu, PMOVSSET_EL0);
@@ -400,7 +400,7 @@ static void kvm_pmu_perf_overflow(struct perf_event *perf_event,
kvm_pmu_counter_increment(vcpu, BIT(idx + 1),
ARMV8_PMUV3_PERFCTR_CHAIN);
- if (kvm_pmu_overflow_status(vcpu)) {
+ if (kvm_pmu_emul_overflow_status(vcpu)) {
kvm_make_request(KVM_REQ_IRQ_PENDING, vcpu);
if (!in_nmi())
diff --git a/arch/arm64/kvm/pmu.c b/arch/arm64/kvm/pmu.c
index 7389d61957fa9..9d4ad4b7bd8d5 100644
--- a/arch/arm64/kvm/pmu.c
+++ b/arch/arm64/kvm/pmu.c
@@ -409,6 +409,14 @@ u64 kvm_pmu_evtyper_mask(struct kvm *kvm)
return mask;
}
+bool kvm_pmu_overflow_status(struct kvm_vcpu *vcpu)
+{
+ if (kvm_pmu_is_partitioned(vcpu->kvm))
+ return kvm_pmu_part_overflow_status(vcpu);
+
+ return kvm_pmu_emul_overflow_status(vcpu);
+}
+
static void kvm_pmu_update_state(struct kvm_vcpu *vcpu)
{
struct kvm_pmu *pmu = &vcpu->arch.pmu;
diff --git a/include/kvm/arm_pmu.h b/include/kvm/arm_pmu.h
index 9800514019630..fcd8e6331943e 100644
--- a/include/kvm/arm_pmu.h
+++ b/include/kvm/arm_pmu.h
@@ -91,6 +91,8 @@ bool kvm_set_pmuserenr(u64 val);
void kvm_vcpu_pmu_restore_guest(struct kvm_vcpu *vcpu);
void kvm_vcpu_pmu_restore_host(struct kvm_vcpu *vcpu);
void kvm_vcpu_pmu_resync_el0(void);
+bool kvm_pmu_emul_overflow_status(struct kvm_vcpu *vcpu);
+bool kvm_pmu_part_overflow_status(struct kvm_vcpu *vcpu);
#define kvm_vcpu_has_pmu(vcpu) \
(vcpu_has_feature(vcpu, KVM_ARM_VCPU_PMU_V3))
--
2.56.1.708.g77570a6a1d-goog
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v10 26/33] KVM: arm64: Multiplex PMCR_EL0.E and MDCR_EL2.HPME for sub-partitioned L1
2026-11-19 18:51 [PATCH v10 00/33] ARM64 PMU Partitioning Colton Lewis
` (24 preceding siblings ...)
2026-11-19 18:52 ` [PATCH v10 25/33] KVM: arm64: Detect overflows for the Partitioned PMU Colton Lewis
@ 2026-11-19 18:52 ` Colton Lewis
2026-11-19 19:16 ` sashiko-bot
2026-11-19 18:52 ` [PATCH v10 27/33] KVM: arm64: Emulate mismatched LP and HLP overflow widths " Colton Lewis
` (6 subsequent siblings)
32 siblings, 1 reply; 67+ messages in thread
From: Colton Lewis @ 2026-11-19 18:52 UTC (permalink / raw)
To: Marc Zyngier, Oliver Upton
Cc: Catalin Marinas, Will Deacon, Mark Rutland, Peter Zijlstra,
Ingo Molnar, Paolo Bonzini, Russell King, Jonathan Corbet,
Fuad Tabba, Joey Gouly, Suzuki K Poulose, Zenghui Yu, Shuah Khan,
James Clark, Alexandru Elisei, Mingwei Zhang, Ganapatrao Kulkarni,
kvmarm, linux-arm-kernel, kvm, linux-perf-users, linux-kselftest,
linux-doc, linux-kernel
When an L1 hypervisor sub-partitions its counters (vHPMN < N_L1, so
kvm_pmu_hyp_counter_mask(vcpu) is non-zero), all N_L1 guest counters lie
below physical MDCR_EL2.HPMN == N_L1 and are gated in hardware by
physical PMCR_EL0.E. Architecturally, however, counters 0..vHPMN-1 and
PMCCNTR_EL0 are gated by vPMCR_EL0.E while L1's hypervisor counters
vHPMN..N_L1-1 are gated independently by vMDCR_EL2.HPME.
Multiplex both domain enables onto physical PMCNTENSET_EL0 and
PMCR_EL0.E whenever L1 sub-partitions its counters (leaving
non-sub-partitioned guests completely untrapped):
- Keep MDCR_EL2.TPMCR and HDFG{R,W}TR_EL2.PMCNTEN enabled when
kvm_pmu_hyp_counter_mask(vcpu) is non-zero.
- Maintain architectural guest PMCR_EL0 and PMCNTENSET_EL0 values in
sys_regs[], set physical PMCR_EL0.E when either vPMCR_EL0.E or
vMDCR_EL2.HPME is set, and program physical PMCNTENSET_EL0 with only
those guest-enabled counters whose domain enable is active.
Signed-off-by: Colton Lewis <coltonlewis@google.com>
---
arch/arm64/kvm/config.c | 6 ++
arch/arm64/kvm/debug.c | 10 ++-
arch/arm64/kvm/pmu-direct.c | 166 +++++++++++++++++++++++++++---------
include/kvm/arm_pmu.h | 1 +
4 files changed, 140 insertions(+), 43 deletions(-)
diff --git a/arch/arm64/kvm/config.c b/arch/arm64/kvm/config.c
index 3220ed6fc9453..4089a144b8f2b 100644
--- a/arch/arm64/kvm/config.c
+++ b/arch/arm64/kvm/config.c
@@ -1801,6 +1801,9 @@ static void __compute_hdfgrtr(struct kvm_vcpu *vcpu)
(__vcpu_sys_reg(vcpu, MDCR_EL2) & MDCR_EL2_TPM))
set |= HDFGRTR_EL2_PMUSERENR_EL0;
+ if (kvm_pmu_hyp_counter_mask(vcpu))
+ set |= HDFGRTR_EL2_PMCNTEN;
+
*vcpu_fgt(vcpu, HDFGRTR_EL2) |= set & ~hdfgrtr_masks.res0;
}
}
@@ -1820,6 +1823,9 @@ static void __compute_hdfgwtr(struct kvm_vcpu *vcpu)
if (is_hyp_ctxt(vcpu))
set |= HDFGWTR_EL2_PMUSERENR_EL0;
+ if (kvm_pmu_hyp_counter_mask(vcpu))
+ set |= HDFGWTR_EL2_PMCNTEN;
+
*vcpu_fgt(vcpu, HDFGWTR_EL2) |= set & ~hdfgwtr_masks.res0;
}
}
diff --git a/arch/arm64/kvm/debug.c b/arch/arm64/kvm/debug.c
index ec9f942b452d8..b89f46cb59955 100644
--- a/arch/arm64/kvm/debug.c
+++ b/arch/arm64/kvm/debug.c
@@ -80,13 +80,17 @@ void kvm_arm_setup_mdcr_el2(struct kvm_vcpu *vcpu)
/*
* Take out the coarse grain traps if we are using
* fine grain traps and enforce counter access with
- * HPMN.
+ * HPMN. Keep TPMCR when L1 sub-partitions its counters
+ * so PMCR_EL0 reads and writes trap for multiplexing.
*/
mdcr &= ~MDCR_EL2_HPMN;
mdcr |= FIELD_PREP(MDCR_EL2_HPMN, nr_guest_cntr);
- if (cpus_have_final_cap(ARM64_HAS_FGT))
- mdcr &= ~(MDCR_EL2_TPM | MDCR_EL2_TPMCR);
+ if (cpus_have_final_cap(ARM64_HAS_FGT)) {
+ mdcr &= ~MDCR_EL2_TPM;
+ if (!kvm_pmu_hyp_counter_mask(vcpu))
+ mdcr &= ~MDCR_EL2_TPMCR;
+ }
}
}
diff --git a/arch/arm64/kvm/pmu-direct.c b/arch/arm64/kvm/pmu-direct.c
index b0491d32b5d7b..725b5fc17533e 100644
--- a/arch/arm64/kvm/pmu-direct.c
+++ b/arch/arm64/kvm/pmu-direct.c
@@ -91,6 +91,82 @@ static u64 kvm_vcpu_pmu_guest_counter_mask(struct kvm_vcpu *vcpu)
static void kvm_pmu_set_guest_owned(struct kvm_vcpu *vcpu);
+static u64 kvm_pmu_part_active_cnten_mask(struct kvm_vcpu *vcpu)
+{
+ u64 guest_mask = kvm_vcpu_pmu_guest_counter_mask(vcpu);
+ u64 hyp_mask = kvm_pmu_hyp_counter_mask(vcpu);
+ u64 pmcr = __vcpu_sys_reg(vcpu, PMCR_EL0);
+ u64 mdcr = __vcpu_sys_reg(vcpu, MDCR_EL2);
+ u64 active = 0;
+
+ if (pmcr & ARMV8_PMU_PMCR_E)
+ active |= guest_mask & ~hyp_mask;
+ if (mdcr & MDCR_EL2_HPME)
+ active |= hyp_mask;
+
+ return active;
+}
+
+static void kvm_pmu_part_sync_hw_enables(struct kvm_vcpu *vcpu)
+{
+ u64 guest_mask = kvm_vcpu_pmu_guest_counter_mask(vcpu);
+ u64 cnten = __vcpu_sys_reg(vcpu, PMCNTENSET_EL0) &
+ kvm_pmu_part_active_cnten_mask(vcpu);
+ u64 pmcr = __vcpu_sys_reg(vcpu, PMCR_EL0);
+ u64 mdcr = __vcpu_sys_reg(vcpu, MDCR_EL2);
+ u64 pmcr_mask = ARMV8_PMU_PMCR_MASK &
+ ~(ARMV8_PMU_PMCR_P | ARMV8_PMU_PMCR_C);
+
+ if (!kvm_has_feat(vcpu->kvm, ID_AA64DFR0_EL1, PMUVer, V3P5))
+ pmcr_mask &= ~ARMV8_PMU_PMCR_LP;
+
+ pmcr &= pmcr_mask;
+ if (mdcr & MDCR_EL2_HPME)
+ pmcr |= ARMV8_PMU_PMCR_E;
+
+ write_sysreg(~cnten & guest_mask, pmcntenclr_el0);
+ write_sysreg(cnten & guest_mask, pmcntenset_el0);
+ write_sysreg(pmcr, pmcr_el0);
+}
+
+static void kvm_pmu_direct_cnten_write(struct kvm_vcpu *vcpu, u64 val, bool set)
+{
+ u64 mask = kvm_vcpu_pmu_guest_counter_mask(vcpu);
+
+ lockdep_assert_preemption_disabled();
+
+ val &= mask;
+ if (set) {
+ __vcpu_rmw_sys_reg(vcpu, PMCNTENSET_EL0, |=, val);
+ if (val)
+ kvm_pmu_set_guest_owned(vcpu);
+ } else {
+ __vcpu_rmw_sys_reg(vcpu, PMCNTENSET_EL0, &=, ~val);
+ }
+
+ if (!kvm_pmu_part_on_cpu(vcpu))
+ return;
+
+ if (vcpu->arch.pmu.multiplex_enables)
+ kvm_pmu_part_sync_hw_enables(vcpu);
+ else if (set)
+ write_sysreg(val, pmcntenset_el0);
+ else
+ write_sysreg(val, pmcntenclr_el0);
+}
+
+static u64 kvm_pmu_direct_cnten_read(struct kvm_vcpu *vcpu)
+{
+ u64 mask = kvm_vcpu_pmu_guest_counter_mask(vcpu);
+
+ lockdep_assert_preemption_disabled();
+
+ if (kvm_pmu_part_on_cpu(vcpu) && !vcpu->arch.pmu.multiplex_enables)
+ return read_sysreg(pmcntenset_el0) & mask;
+
+ return __vcpu_sys_reg(vcpu, PMCNTENSET_EL0) & mask;
+}
+
static void kvm_pmu_direct_pmcr_write(struct kvm_vcpu *vcpu, u64 val)
{
bool reset_p = val & ARMV8_PMU_PMCR_P;
@@ -133,7 +209,11 @@ static void kvm_pmu_direct_pmcr_write(struct kvm_vcpu *vcpu, u64 val)
write_pmevcntrn(i, 0);
}
- write_sysreg(val & ~(ARMV8_PMU_PMCR_P | ARMV8_PMU_PMCR_C), pmcr_el0);
+ if (vcpu->arch.pmu.multiplex_enables)
+ kvm_pmu_part_sync_hw_enables(vcpu);
+ else
+ write_sysreg(val & ~(ARMV8_PMU_PMCR_P | ARMV8_PMU_PMCR_C),
+ pmcr_el0);
}
static u64 kvm_pmu_direct_pmcr_read(struct kvm_vcpu *vcpu)
@@ -147,6 +227,10 @@ static u64 kvm_pmu_direct_pmcr_read(struct kvm_vcpu *vcpu)
return kvm_vcpu_read_pmcr(vcpu);
pmcr = read_sysreg(pmcr_el0);
+ if (vcpu->arch.pmu.multiplex_enables) {
+ pmcr &= ~ARMV8_PMU_PMCR_MASK;
+ pmcr |= __vcpu_sys_reg(vcpu, PMCR_EL0) & ARMV8_PMU_PMCR_MASK;
+ }
if (!kvm_has_feat(vcpu->kvm, ID_AA64DFR0_EL1, PMUVer, V3P5))
pmcr &= ~ARMV8_PMU_PMCR_LP;
@@ -232,6 +316,7 @@ static u64 kvm_pmu_direct_cntr_read(struct kvm_vcpu *vcpu, u8 idx)
return read_pmevcntrn(idx);
}
+
static void kvm_pmu_direct_cntr_write(struct kvm_vcpu *vcpu, u8 idx, u64 val)
{
lockdep_assert_preemption_disabled();
@@ -273,10 +358,7 @@ u64 kvm_pmu_direct_reg_read(struct kvm_vcpu *vcpu, unsigned int reg)
case PMEVTYPER0_EL0 ... PMCCFILTR_EL0:
return __vcpu_sys_reg(vcpu, reg);
case PMCNTENSET_EL0:
- if (kvm_pmu_part_on_cpu(vcpu))
- return read_sysreg(pmcntenset_el0) &
- kvm_vcpu_pmu_guest_counter_mask(vcpu);
- return __vcpu_sys_reg(vcpu, reg);
+ return kvm_pmu_direct_cnten_read(vcpu);
case PMINTENSET_EL1:
if (kvm_pmu_part_on_cpu(vcpu))
return read_sysreg(pmintenset_el1) &
@@ -410,18 +492,7 @@ void kvm_pmu_direct_reg_write(struct kvm_vcpu *vcpu, unsigned int reg,
kvm_pmu_apply_single_event_filter(vcpu, reg - PMEVTYPER0_EL0);
break;
case PMCNTENSET_EL0:
- val &= kvm_vcpu_pmu_guest_counter_mask(vcpu);
- if (set) {
- __vcpu_rmw_sys_reg(vcpu, PMCNTENSET_EL0, |=, val);
- if (val)
- kvm_pmu_set_guest_owned(vcpu);
- if (kvm_pmu_part_on_cpu(vcpu))
- write_sysreg(val, pmcntenset_el0);
- } else {
- __vcpu_rmw_sys_reg(vcpu, PMCNTENSET_EL0, &=, ~val);
- if (kvm_pmu_part_on_cpu(vcpu))
- write_sysreg(val, pmcntenclr_el0);
- }
+ kvm_pmu_direct_cnten_write(vcpu, val, set);
break;
case PMINTENSET_EL1:
val &= kvm_vcpu_pmu_guest_counter_mask(vcpu);
@@ -545,6 +616,7 @@ static void __kvm_pmu_load(struct kvm_vcpu *vcpu, bool reserve_counters)
isb();
vcpu->arch.pmu.loaded_on_cpu = true;
+ vcpu->arch.pmu.multiplex_enables = !!kvm_pmu_hyp_counter_mask(vcpu);
kvm_arm_setup_mdcr_el2(vcpu);
isb();
@@ -568,14 +640,6 @@ static void __kvm_pmu_load(struct kvm_vcpu *vcpu, bool reserve_counters)
write_sysreg(val, pmuserenr_el0);
}
- /* Save only the stateful writable bits. */
- val = __vcpu_sys_reg(vcpu, PMCR_EL0);
- mask = ARMV8_PMU_PMCR_MASK &
- ~(ARMV8_PMU_PMCR_P | ARMV8_PMU_PMCR_C);
- if (!kvm_has_feat(vcpu->kvm, ID_AA64DFR0_EL1, PMUVer, V3P5))
- mask &= ~ARMV8_PMU_PMCR_LP;
- write_sysreg(val & mask, pmcr_el0);
-
/*
* When handling these:
* 1. Apply only the bits for guest counters (indicated by mask)
@@ -591,9 +655,23 @@ static void __kvm_pmu_load(struct kvm_vcpu *vcpu, bool reserve_counters)
write_sysreg(mask, pmovsclr_el0);
isb();
- val = __vcpu_sys_reg(vcpu, PMCNTENSET_EL0);
- write_sysreg(val & mask, pmcntenset_el0);
- write_sysreg(~val & mask, pmcntenclr_el0);
+ if (vcpu->arch.pmu.multiplex_enables) {
+ kvm_pmu_part_sync_hw_enables(vcpu);
+ } else {
+ u64 pmcr_mask = ARMV8_PMU_PMCR_MASK &
+ ~(ARMV8_PMU_PMCR_P | ARMV8_PMU_PMCR_C);
+
+ if (!kvm_has_feat(vcpu->kvm, ID_AA64DFR0_EL1, PMUVer, V3P5))
+ pmcr_mask &= ~ARMV8_PMU_PMCR_LP;
+
+ /* Save only the stateful writable bits. */
+ val = __vcpu_sys_reg(vcpu, PMCR_EL0);
+ write_sysreg(val & pmcr_mask, pmcr_el0);
+
+ val = __vcpu_sys_reg(vcpu, PMCNTENSET_EL0);
+ write_sysreg(val & mask, pmcntenset_el0);
+ write_sysreg(~val & mask, pmcntenclr_el0);
+ }
val = __vcpu_sys_reg(vcpu, PMINTENSET_EL1);
write_sysreg(val & mask, pmintenset_el1);
@@ -657,8 +735,10 @@ static void __kvm_pmu_put(struct kvm_vcpu *vcpu, bool release_counters)
daif = local_daif_save();
/* Mask these to only save the guest relevant bits. */
- val = read_sysreg(pmcntenset_el0);
- __vcpu_assign_sys_reg(vcpu, PMCNTENSET_EL0, val & mask);
+ if (!vcpu->arch.pmu.multiplex_enables) {
+ val = read_sysreg(pmcntenset_el0);
+ __vcpu_assign_sys_reg(vcpu, PMCNTENSET_EL0, val & mask);
+ }
val = read_sysreg(pmintenset_el1);
__vcpu_assign_sys_reg(vcpu, PMINTENSET_EL1, val & mask);
@@ -690,12 +770,14 @@ static void __kvm_pmu_put(struct kvm_vcpu *vcpu, bool release_counters)
write_sysreg(ARMV8_PMU_USERENR_MASK, pmuserenr_el0);
}
- val = read_sysreg(pmcr_el0);
- pmcr_mask = ARMV8_PMU_PMCR_MASK;
- if (!kvm_has_feat(vcpu->kvm, ID_AA64DFR0_EL1, PMUVer, V3P5))
- pmcr_mask &= ~ARMV8_PMU_PMCR_LP;
- __vcpu_rmw_sys_reg(vcpu, PMCR_EL0, &=, ~ARMV8_PMU_PMCR_MASK);
- __vcpu_rmw_sys_reg(vcpu, PMCR_EL0, |=, val & pmcr_mask);
+ if (!vcpu->arch.pmu.multiplex_enables) {
+ pmcr_mask = ARMV8_PMU_PMCR_MASK;
+ if (!kvm_has_feat(vcpu->kvm, ID_AA64DFR0_EL1, PMUVer, V3P5))
+ pmcr_mask &= ~ARMV8_PMU_PMCR_LP;
+ val = read_sysreg(pmcr_el0);
+ __vcpu_rmw_sys_reg(vcpu, PMCR_EL0, &=, ~ARMV8_PMU_PMCR_MASK);
+ __vcpu_rmw_sys_reg(vcpu, PMCR_EL0, |=, val & pmcr_mask);
+ }
val = ARMV8_PMU_PMCR_LC;
if (pmu->pmuver >= ID_AA64DFR0_EL1_PMUVer_V3P5)
@@ -710,6 +792,7 @@ static void __kvm_pmu_put(struct kvm_vcpu *vcpu, bool release_counters)
write_sysreg(val & mask, pmovsclr_el0);
vcpu->arch.pmu.loaded_on_cpu = false;
+ vcpu->arch.pmu.multiplex_enables = false;
kvm_arm_setup_mdcr_el2(vcpu);
isb();
@@ -741,9 +824,9 @@ void kvm_pmu_put(struct kvm_vcpu *vcpu)
* kvm_pmu_reload_part() - Reload partitioned PMU traps, filters, and state
* @vcpu: Pointer to struct kvm_vcpu
*
- * Refresh MDCR_EL2 trap bits, event filters, and hardware register state
- * without releasing the host counter reservation when the partitioned PMU is
- * already loaded on the CPU.
+ * Refresh fine-grained traps, MDCR_EL2 trap bits, event filters, and
+ * enable multiplexing without releasing the host counter reservation
+ * when the partitioned PMU is already loaded on the CPU.
*/
void kvm_pmu_reload_part(struct kvm_vcpu *vcpu)
{
@@ -864,7 +947,10 @@ bool kvm_pmu_part_overflow_status(struct kvm_vcpu *vcpu)
write_sysreg(hw_ovf, pmovsclr_el0);
}
pmint = read_pmintenset();
- pmcr = read_pmcr();
+ if (vcpu->arch.pmu.multiplex_enables)
+ pmcr = __vcpu_sys_reg(vcpu, PMCR_EL0);
+ else
+ pmcr = read_pmcr();
} else {
pmint = __vcpu_sys_reg(vcpu, PMINTENSET_EL1);
pmcr = __vcpu_sys_reg(vcpu, PMCR_EL0);
diff --git a/include/kvm/arm_pmu.h b/include/kvm/arm_pmu.h
index fcd8e6331943e..9e5fcbe2ccbb1 100644
--- a/include/kvm/arm_pmu.h
+++ b/include/kvm/arm_pmu.h
@@ -45,6 +45,7 @@ struct kvm_pmu {
int irq_num;
bool created;
bool loaded_on_cpu;
+ bool multiplex_enables;
enum vcpu_pmu_register_access access;
};
--
2.56.1.708.g77570a6a1d-goog
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v10 27/33] KVM: arm64: Emulate mismatched LP and HLP overflow widths for sub-partitioned L1
2026-11-19 18:51 [PATCH v10 00/33] ARM64 PMU Partitioning Colton Lewis
` (25 preceding siblings ...)
2026-11-19 18:52 ` [PATCH v10 26/33] KVM: arm64: Multiplex PMCR_EL0.E and MDCR_EL2.HPME for sub-partitioned L1 Colton Lewis
@ 2026-11-19 18:52 ` Colton Lewis
2026-11-19 19:26 ` sashiko-bot
2026-11-19 18:52 ` [PATCH v10 28/33] KVM: arm64: Emulate trapped PMSWINC_EL0 writes for Partitioned PMU Colton Lewis
` (5 subsequent siblings)
32 siblings, 1 reply; 67+ messages in thread
From: Colton Lewis @ 2026-11-19 18:52 UTC (permalink / raw)
To: Marc Zyngier, Oliver Upton
Cc: Catalin Marinas, Will Deacon, Mark Rutland, Peter Zijlstra,
Ingo Molnar, Paolo Bonzini, Russell King, Jonathan Corbet,
Fuad Tabba, Joey Gouly, Suzuki K Poulose, Zenghui Yu, Shuah Khan,
James Clark, Alexandru Elisei, Mingwei Zhang, Ganapatrao Kulkarni,
kvmarm, linux-arm-kernel, kvm, linux-perf-users, linux-kselftest,
linux-doc, linux-kernel
When an L1 hypervisor sub-partitions its counters (vHPMN < N_L1) on
FEAT_PMUv3p5, all N_L1 guest counters lie below physical MDCR_EL2.HPMN
and share a single physical overflow width bit (PMCR_EL0.LP), whereas
architecturally counters 0..vHPMN-1 use vPMCR_EL0.LP and hypervisor
counters vHPMN..N_L1-1 use vMDCR_EL2.HLP.
When vPMCR_EL0.LP != vMDCR_EL2.HLP (and both sub-ranges are non-empty):
- Force physical PMCR_EL0.LP to 1 (64-bit hardware overflow) and bias
each counter in the 32-bit sub-range (ovf32_mask) to
0xffffffff00000000 | lower_32_bits(val) so hardware overflows after
2^32 events, storing the upper-32-bit architectural offset in
vcpu->arch.pmu.ovf32_offset[idx].
- Trap PMEVCNTR<n>_EL0 reads and writes via HDFG{R,W}TR_EL2.PMEVCNTRn_EL0
while ovf32_mask is non-zero to translate between biased hardware
values and architectural 64-bit values, and trigger KVM_REQ_RELOAD_PMU
when guest PMCR_EL0.LP or MDCR_EL2.HLP changes.
- Defer re-biasing until non-interrupt context (counter reads,
kvm_pmu_part_overflow_status(), or __kvm_pmu_put()) so NMI/IRQ
overflow handlers remain lockless while phys_val + ovf32_offset[idx]
stays exact modulo 2^64 across wraparound.
Signed-off-by: Colton Lewis <coltonlewis@google.com>
---
arch/arm64/kvm/config.c | 4 +
arch/arm64/kvm/pmu-direct.c | 187 +++++++++++++++++++++++++++++-------
arch/arm64/kvm/sys_regs.c | 3 +-
include/kvm/arm_pmu.h | 8 ++
4 files changed, 164 insertions(+), 38 deletions(-)
diff --git a/arch/arm64/kvm/config.c b/arch/arm64/kvm/config.c
index 4089a144b8f2b..a523dde1b57a7 100644
--- a/arch/arm64/kvm/config.c
+++ b/arch/arm64/kvm/config.c
@@ -1803,6 +1803,8 @@ static void __compute_hdfgrtr(struct kvm_vcpu *vcpu)
if (kvm_pmu_hyp_counter_mask(vcpu))
set |= HDFGRTR_EL2_PMCNTEN;
+ if (kvm_pmu_part_ovf32_mask(vcpu))
+ set |= HDFGRTR_EL2_PMEVCNTRn_EL0;
*vcpu_fgt(vcpu, HDFGRTR_EL2) |= set & ~hdfgrtr_masks.res0;
}
@@ -1825,6 +1827,8 @@ static void __compute_hdfgwtr(struct kvm_vcpu *vcpu)
if (kvm_pmu_hyp_counter_mask(vcpu))
set |= HDFGWTR_EL2_PMCNTEN;
+ if (kvm_pmu_part_ovf32_mask(vcpu))
+ set |= HDFGWTR_EL2_PMEVCNTRn_EL0;
*vcpu_fgt(vcpu, HDFGWTR_EL2) |= set & ~hdfgwtr_masks.res0;
}
diff --git a/arch/arm64/kvm/pmu-direct.c b/arch/arm64/kvm/pmu-direct.c
index 725b5fc17533e..b7bec06b36b0a 100644
--- a/arch/arm64/kvm/pmu-direct.c
+++ b/arch/arm64/kvm/pmu-direct.c
@@ -91,6 +91,41 @@ static u64 kvm_vcpu_pmu_guest_counter_mask(struct kvm_vcpu *vcpu)
static void kvm_pmu_set_guest_owned(struct kvm_vcpu *vcpu);
+/**
+ * kvm_pmu_part_ovf32_mask() - Counters requiring 32-bit overflow emulation
+ * @vcpu: Pointer to vcpu struct
+ *
+ * When L1 sub-partitions its counters (vHPMN < N_L1) on FEAT_PMUv3p5 and
+ * configures mismatched overflow widths (vPMCR_EL0.LP != vMDCR_EL2.HLP),
+ * physical PMCR_EL0.LP is forced to 1 and the 32-bit sub-range is biased
+ * 2^32 events below 64-bit overflow.
+ *
+ * Return: Bitmask of general-purpose counters requiring 32-bit overflow bias
+ */
+u64 kvm_pmu_part_ovf32_mask(struct kvm_vcpu *vcpu)
+{
+ u64 hyp_mask = kvm_pmu_hyp_counter_mask(vcpu);
+ u64 pmcr, mdcr, ev_mask;
+ bool lp, hlp;
+
+ if (!hyp_mask ||
+ !kvm_has_feat(vcpu->kvm, ID_AA64DFR0_EL1, PMUVer, V3P5))
+ return 0;
+
+ pmcr = __vcpu_sys_reg(vcpu, PMCR_EL0);
+ mdcr = __vcpu_sys_reg(vcpu, MDCR_EL2);
+ lp = !!(pmcr & ARMV8_PMU_PMCR_LP);
+ hlp = !!(mdcr & MDCR_EL2_HLP);
+ if (lp == hlp)
+ return 0;
+
+ ev_mask = kvm_vcpu_pmu_guest_counter_mask(vcpu) &
+ ~BIT_ULL(ARMV8_PMU_CYCLE_IDX);
+ if (!(ev_mask & ~hyp_mask))
+ return 0;
+
+ return lp ? hyp_mask : (ev_mask & ~hyp_mask);
+}
static u64 kvm_pmu_part_active_cnten_mask(struct kvm_vcpu *vcpu)
{
u64 guest_mask = kvm_vcpu_pmu_guest_counter_mask(vcpu);
@@ -110,6 +145,8 @@ static u64 kvm_pmu_part_active_cnten_mask(struct kvm_vcpu *vcpu)
static void kvm_pmu_part_sync_hw_enables(struct kvm_vcpu *vcpu)
{
u64 guest_mask = kvm_vcpu_pmu_guest_counter_mask(vcpu);
+ u64 ev_mask = guest_mask & ~BIT_ULL(ARMV8_PMU_CYCLE_IDX);
+ u64 hyp_mask = kvm_pmu_hyp_counter_mask(vcpu);
u64 cnten = __vcpu_sys_reg(vcpu, PMCNTENSET_EL0) &
kvm_pmu_part_active_cnten_mask(vcpu);
u64 pmcr = __vcpu_sys_reg(vcpu, PMCR_EL0);
@@ -123,12 +160,109 @@ static void kvm_pmu_part_sync_hw_enables(struct kvm_vcpu *vcpu)
pmcr &= pmcr_mask;
if (mdcr & MDCR_EL2_HPME)
pmcr |= ARMV8_PMU_PMCR_E;
+ if (kvm_has_feat(vcpu->kvm, ID_AA64DFR0_EL1, PMUVer, V3P5)) {
+ if (ev_mask && !(ev_mask & ~hyp_mask)) {
+ if (mdcr & MDCR_EL2_HLP)
+ pmcr |= ARMV8_PMU_PMCR_LP;
+ else
+ pmcr &= ~ARMV8_PMU_PMCR_LP;
+ } else if (vcpu->arch.pmu.ovf32_mask) {
+ pmcr |= ARMV8_PMU_PMCR_LP;
+ }
+ }
write_sysreg(~cnten & guest_mask, pmcntenclr_el0);
write_sysreg(cnten & guest_mask, pmcntenset_el0);
write_sysreg(pmcr, pmcr_el0);
}
+static void kvm_pmu_part_write_hw_cntr(struct kvm_vcpu *vcpu, u8 idx, u64 val)
+{
+ if (idx == ARMV8_PMU_CYCLE_IDX) {
+ write_pmccntr(val);
+ return;
+ }
+
+ if (vcpu->arch.pmu.ovf32_mask & BIT_ULL(idx)) {
+ vcpu->arch.pmu.ovf32_offset[idx] =
+ (val & GENMASK_ULL(63, 32)) + BIT_ULL(32);
+ val = GENMASK_ULL(63, 32) | lower_32_bits(val);
+ }
+
+ write_pmevcntrn(idx, val);
+}
+
+static void kvm_pmu_part_rebias_single(struct kvm_vcpu *vcpu, u8 idx)
+{
+ u64 phys_val, arch_val, cnten;
+
+ phys_val = read_pmevcntrn(idx);
+ if (upper_32_bits(phys_val) == U32_MAX)
+ return;
+
+ cnten = read_sysreg(pmcntenset_el0) & BIT_ULL(idx);
+ if (cnten) {
+ write_sysreg(cnten, pmcntenclr_el0);
+ isb();
+ }
+
+ if (read_sysreg(pmovsset_el0) & BIT_ULL(idx)) {
+ __vcpu_rmw_sys_reg(vcpu, PMOVSSET_EL0, |=, BIT_ULL(idx));
+ write_sysreg(BIT_ULL(idx), pmovsclr_el0);
+ }
+
+ phys_val = read_pmevcntrn(idx);
+ arch_val = phys_val + vcpu->arch.pmu.ovf32_offset[idx];
+ kvm_pmu_part_write_hw_cntr(vcpu, idx, arch_val);
+
+ if (cnten) {
+ isb();
+ write_sysreg(cnten, pmcntenset_el0);
+ }
+}
+
+static void kvm_pmu_part_rebias_ovf32(struct kvm_vcpu *vcpu)
+{
+ unsigned long mask = vcpu->arch.pmu.ovf32_mask;
+ u8 i;
+
+ for_each_set_bit(i, &mask, ARMV8_PMU_MAX_GENERAL_COUNTERS)
+ kvm_pmu_part_rebias_single(vcpu, i);
+}
+
+static void kvm_pmu_direct_cntr_write(struct kvm_vcpu *vcpu, u8 idx, u64 val)
+{
+ lockdep_assert_preemption_disabled();
+
+ __vcpu_assign_sys_reg(vcpu, PMEVCNTR0_EL0 + idx, val);
+ if (kvm_pmu_part_on_cpu(vcpu))
+ kvm_pmu_part_write_hw_cntr(vcpu, idx, val);
+}
+
+static u64 kvm_pmu_direct_cntr_read(struct kvm_vcpu *vcpu, u8 idx)
+{
+ lockdep_assert_preemption_disabled();
+
+ if (!kvm_pmu_part_on_cpu(vcpu))
+ return __vcpu_sys_reg(vcpu, PMEVCNTR0_EL0 + idx);
+
+ if (idx == ARMV8_PMU_CYCLE_IDX)
+ return read_pmccntr();
+
+ if (vcpu->arch.pmu.ovf32_mask & BIT_ULL(idx)) {
+ unsigned long flags;
+ u64 val;
+
+ local_irq_save(flags);
+ kvm_pmu_part_rebias_single(vcpu, idx);
+ val = read_pmevcntrn(idx) + vcpu->arch.pmu.ovf32_offset[idx];
+ local_irq_restore(flags);
+ return val;
+ }
+
+ return read_pmevcntrn(idx);
+}
+
static void kvm_pmu_direct_cnten_write(struct kvm_vcpu *vcpu, u64 val, bool set)
{
u64 mask = kvm_vcpu_pmu_guest_counter_mask(vcpu);
@@ -172,11 +306,12 @@ static void kvm_pmu_direct_pmcr_write(struct kvm_vcpu *vcpu, u64 val)
bool reset_p = val & ARMV8_PMU_PMCR_P;
bool reset_c = val & ARMV8_PMU_PMCR_C;
unsigned long mask;
- u64 pmcr_mask;
+ u64 old, pmcr_mask;
int i;
lockdep_assert_preemption_disabled();
+ old = __vcpu_sys_reg(vcpu, PMCR_EL0);
if (!kvm_has_feat(vcpu->kvm, ID_AA64DFR0_EL1, PMUVer, V3P5))
val &= ~ARMV8_PMU_PMCR_LP;
@@ -195,6 +330,9 @@ static void kvm_pmu_direct_pmcr_write(struct kvm_vcpu *vcpu, u64 val)
__vcpu_assign_sys_reg(vcpu, PMEVCNTR0_EL0 + i, 0);
}
+ if (((old ^ val) & ARMV8_PMU_PMCR_LP) && kvm_pmu_hyp_counter_mask(vcpu))
+ kvm_make_request(KVM_REQ_RELOAD_PMU, vcpu);
+
if (val & ARMV8_PMU_PMCR_E)
kvm_pmu_set_guest_owned(vcpu);
@@ -206,7 +344,7 @@ static void kvm_pmu_direct_pmcr_write(struct kvm_vcpu *vcpu, u64 val)
if (reset_p) {
for_each_set_bit(i, &mask, ARMV8_PMU_MAX_GENERAL_COUNTERS)
- write_pmevcntrn(i, 0);
+ kvm_pmu_part_write_hw_cntr(vcpu, i, 0);
}
if (vcpu->arch.pmu.multiplex_enables)
@@ -304,33 +442,6 @@ static void kvm_pmu_set_guest_counters(struct arm_pmu *cpu_pmu, u64 guest_mask)
local_irq_restore(flags);
}
-static u64 kvm_pmu_direct_cntr_read(struct kvm_vcpu *vcpu, u8 idx)
-{
- lockdep_assert_preemption_disabled();
-
- if (!kvm_pmu_part_on_cpu(vcpu))
- return __vcpu_sys_reg(vcpu, PMEVCNTR0_EL0 + idx);
-
- if (idx == ARMV8_PMU_CYCLE_IDX)
- return read_pmccntr();
-
- return read_pmevcntrn(idx);
-}
-
-static void kvm_pmu_direct_cntr_write(struct kvm_vcpu *vcpu, u8 idx, u64 val)
-{
- lockdep_assert_preemption_disabled();
-
- __vcpu_assign_sys_reg(vcpu, PMEVCNTR0_EL0 + idx, val);
- if (!kvm_pmu_part_on_cpu(vcpu))
- return;
-
- if (idx == ARMV8_PMU_CYCLE_IDX)
- write_pmccntr(val);
- else
- write_pmevcntrn(idx, val);
-}
-
/**
* kvm_pmu_direct_reg_read() - Read a guest PMU register for Partitioned PMU
* @vcpu: Pointer to vcpu struct
@@ -617,6 +728,7 @@ static void __kvm_pmu_load(struct kvm_vcpu *vcpu, bool reserve_counters)
vcpu->arch.pmu.loaded_on_cpu = true;
vcpu->arch.pmu.multiplex_enables = !!kvm_pmu_hyp_counter_mask(vcpu);
+ vcpu->arch.pmu.ovf32_mask = kvm_pmu_part_ovf32_mask(vcpu);
kvm_arm_setup_mdcr_el2(vcpu);
isb();
@@ -624,11 +736,7 @@ static void __kvm_pmu_load(struct kvm_vcpu *vcpu, bool reserve_counters)
for_each_set_bit(i, &guest_counters, ARMPMU_MAX_HWEVENTS) {
val = __vcpu_sys_reg(vcpu, PMEVCNTR0_EL0 + i);
-
- if (i == ARMV8_PMU_CYCLE_IDX)
- write_pmccntr(val);
- else
- write_pmevcntrn(i, val);
+ kvm_pmu_part_write_hw_cntr(vcpu, i, val);
}
if (!(vcpu->arch.mdcr_el2 & MDCR_EL2_TPM)) {
@@ -754,6 +862,8 @@ static void __kvm_pmu_put(struct kvm_vcpu *vcpu, bool release_counters)
write_pmccntr(0);
} else {
val = read_pmevcntrn(i);
+ if (vcpu->arch.pmu.ovf32_mask & BIT_ULL(i))
+ val += vcpu->arch.pmu.ovf32_offset[i];
write_pmevcntrn(i, 0);
}
@@ -793,6 +903,7 @@ static void __kvm_pmu_put(struct kvm_vcpu *vcpu, bool release_counters)
vcpu->arch.pmu.loaded_on_cpu = false;
vcpu->arch.pmu.multiplex_enables = false;
+ vcpu->arch.pmu.ovf32_mask = 0;
kvm_arm_setup_mdcr_el2(vcpu);
isb();
@@ -824,9 +935,9 @@ void kvm_pmu_put(struct kvm_vcpu *vcpu)
* kvm_pmu_reload_part() - Reload partitioned PMU traps, filters, and state
* @vcpu: Pointer to struct kvm_vcpu
*
- * Refresh fine-grained traps, MDCR_EL2 trap bits, event filters, and
- * enable multiplexing without releasing the host counter reservation
- * when the partitioned PMU is already loaded on the CPU.
+ * Refresh fine-grained traps, MDCR_EL2 trap bits, event filters, enable
+ * multiplexing, and 32-bit overflow bias without releasing the host counter
+ * reservation when the partitioned PMU is already loaded on the CPU.
*/
void kvm_pmu_reload_part(struct kvm_vcpu *vcpu)
{
@@ -946,6 +1057,8 @@ bool kvm_pmu_part_overflow_status(struct kvm_vcpu *vcpu)
__vcpu_rmw_sys_reg(vcpu, PMOVSSET_EL0, |=, hw_ovf);
write_sysreg(hw_ovf, pmovsclr_el0);
}
+ if (!in_interrupt() && vcpu->arch.pmu.ovf32_mask)
+ kvm_pmu_part_rebias_ovf32(vcpu);
pmint = read_pmintenset();
if (vcpu->arch.pmu.multiplex_enables)
pmcr = __vcpu_sys_reg(vcpu, PMCR_EL0);
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 6706f57aef8d5..24dd878fb58c4 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -3185,7 +3185,8 @@ static bool access_mdcr(struct kvm_vcpu *vcpu,
* affected by MDCR_EL2.
*/
if ((old ^ val) & (MDCR_EL2_HPME | MDCR_EL2_HPMN | MDCR_EL2_HPMD |
- MDCR_EL2_HCCD | MDCR_EL2_TPM | MDCR_EL2_TPMCR))
+ MDCR_EL2_HCCD | MDCR_EL2_HLP | MDCR_EL2_TPM |
+ MDCR_EL2_TPMCR))
kvm_make_request(KVM_REQ_RELOAD_PMU, vcpu);
return true;
diff --git a/include/kvm/arm_pmu.h b/include/kvm/arm_pmu.h
index 9e5fcbe2ccbb1..13e9230c9c714 100644
--- a/include/kvm/arm_pmu.h
+++ b/include/kvm/arm_pmu.h
@@ -42,6 +42,8 @@ struct kvm_pmu {
struct irq_work overflow_work;
struct kvm_pmu_events events;
struct kvm_pmc pmc[KVM_ARMV8_PMU_MAX_COUNTERS];
+ u64 ovf32_offset[ARMV8_PMU_MAX_GENERAL_COUNTERS];
+ u64 ovf32_mask;
int irq_num;
bool created;
bool loaded_on_cpu;
@@ -107,6 +109,7 @@ bool kvm_pmu_is_partitioned(struct kvm *kvm);
u64 kvm_pmu_direct_reg_read(struct kvm_vcpu *vcpu, unsigned int reg);
void kvm_pmu_direct_reg_write(struct kvm_vcpu *vcpu, unsigned int reg,
u64 val, bool set);
+u64 kvm_pmu_part_ovf32_mask(struct kvm_vcpu *vcpu);
void kvm_pmu_reload_part(struct kvm_vcpu *vcpu);
u64 kvm_pmu_guest_counter_mask(void);
void kvm_pmu_load(struct kvm_vcpu *vcpu);
@@ -171,6 +174,11 @@ static inline u64 kvm_pmu_direct_reg_read(struct kvm_vcpu *vcpu,
static inline void kvm_pmu_direct_reg_write(struct kvm_vcpu *vcpu,
unsigned int reg, u64 val,
bool set) {}
+
+static inline u64 kvm_pmu_part_ovf32_mask(struct kvm_vcpu *vcpu)
+{
+ return 0;
+}
static inline void kvm_pmu_reload_part(struct kvm_vcpu *vcpu) {}
static inline void kvm_pmu_load(struct kvm_vcpu *vcpu) {}
static inline void kvm_pmu_put(struct kvm_vcpu *vcpu) {}
--
2.56.1.708.g77570a6a1d-goog
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v10 28/33] KVM: arm64: Emulate trapped PMSWINC_EL0 writes for Partitioned PMU
2026-11-19 18:51 [PATCH v10 00/33] ARM64 PMU Partitioning Colton Lewis
` (26 preceding siblings ...)
2026-11-19 18:52 ` [PATCH v10 27/33] KVM: arm64: Emulate mismatched LP and HLP overflow widths " Colton Lewis
@ 2026-11-19 18:52 ` Colton Lewis
2026-11-19 19:13 ` sashiko-bot
2026-11-19 18:52 ` [PATCH v10 29/33] KVM: arm64: Forward FGT traps for PMXEVTYPER_EL0 and PMXEVCNTR_EL0 Colton Lewis
` (4 subsequent siblings)
32 siblings, 1 reply; 67+ messages in thread
From: Colton Lewis @ 2026-11-19 18:52 UTC (permalink / raw)
To: Marc Zyngier, Oliver Upton
Cc: Catalin Marinas, Will Deacon, Mark Rutland, Peter Zijlstra,
Ingo Molnar, Paolo Bonzini, Russell King, Jonathan Corbet,
Fuad Tabba, Joey Gouly, Suzuki K Poulose, Zenghui Yu, Shuah Khan,
James Clark, Alexandru Elisei, Mingwei Zhang, Ganapatrao Kulkarni,
kvmarm, linux-arm-kernel, kvm, linux-perf-users, linux-kselftest,
linux-doc, linux-kernel
Even with a Partitioned PMU, guest writes to PMSWINC_EL0 trap to EL2 in
four cases:
1. Hosts without FEAT_FGT, where MDCR_EL2.TPM remains set.
2. Before lazy PMU activation (VCPU_PMU_ACCESS_FREE, MDCR_EL2.TPM == 1).
3. Nested L2 execution when L1 sub-partitions counters (vHPMN < N_L1),
where MDCR_EL2.TPM is forced to protect L1's hypervisor counters.
4. Nested L1 execution when L1 sub-partitions counters (vHPMN < N_L1),
where HDFGWTR_EL2.PMSWINC_EL0 must trap software increments so an
even counter at vHPMN - 1 cannot chain across the sub-partition
boundary into L1 counter vHPMN in hardware, and so ovf32_mask
counter biasing is preserved.
Trapped PMSWINC_EL0 writes cannot simply execute MSR PMSWINC_EL0 at
physical EL2 because L0 runs with physical MDCR_EL2.HPMD == 1 and clears
PMEVTYPER<n>_EL0.NSH on guest counters, causing hardware to ignore the
software increment (and when not loaded on the CPU, guest counters live
only in sys_regs[]).
Update kvm_pmu_counter_increment() and kvm_pmu_counter_can_chain() in
arch/arm64/kvm/pmu-emul.c to check per-counter enablement
(kvm_pmu_counter_is_enabled(), covering both PMCR_EL0.E and
MDCR_EL2.HPME), current guest EL filtering for SW_INCR, and
nr_pmu_counters / sub-partition chaining boundaries. Reuse
kvm_pmu_software_increment() from kvm_pmu_direct_swinc() by flushing
live hardware state if loaded, applying the software increment to
sys_regs[], and reloading hardware.
Signed-off-by: Colton Lewis <coltonlewis@google.com>
---
arch/arm64/kvm/config.c | 2 +-
arch/arm64/kvm/pmu-direct.c | 34 ++++++++++
arch/arm64/kvm/pmu-emul.c | 120 +++++++++++++++++++++---------------
arch/arm64/kvm/sys_regs.c | 5 +-
include/kvm/arm_pmu.h | 3 +
5 files changed, 113 insertions(+), 51 deletions(-)
diff --git a/arch/arm64/kvm/config.c b/arch/arm64/kvm/config.c
index a523dde1b57a7..0ec74e45fdbf6 100644
--- a/arch/arm64/kvm/config.c
+++ b/arch/arm64/kvm/config.c
@@ -1826,7 +1826,7 @@ static void __compute_hdfgwtr(struct kvm_vcpu *vcpu)
set |= HDFGWTR_EL2_PMUSERENR_EL0;
if (kvm_pmu_hyp_counter_mask(vcpu))
- set |= HDFGWTR_EL2_PMCNTEN;
+ set |= HDFGWTR_EL2_PMCNTEN | HDFGWTR_EL2_PMSWINC_EL0;
if (kvm_pmu_part_ovf32_mask(vcpu))
set |= HDFGWTR_EL2_PMEVCNTRn_EL0;
diff --git a/arch/arm64/kvm/pmu-direct.c b/arch/arm64/kvm/pmu-direct.c
index b7bec06b36b0a..6382233e54c71 100644
--- a/arch/arm64/kvm/pmu-direct.c
+++ b/arch/arm64/kvm/pmu-direct.c
@@ -931,6 +931,40 @@ void kvm_pmu_put(struct kvm_vcpu *vcpu)
__kvm_pmu_put(vcpu, release);
}
+/**
+ * kvm_pmu_direct_swinc() - Emulate trapped PMSWINC_EL0 writes
+ * @vcpu: Pointer to vcpu struct
+ * @val: Accessible counter bitmask written to PMSWINC_EL0
+ *
+ * Executing MSR PMSWINC_EL0 at physical EL2 is ignored by hardware because
+ * physical MDCR_EL2.HPMD is set and guest PMEVTYPER<n>_EL0.NSH is cleared.
+ * Flush live hardware state if loaded, apply kvm_pmu_software_increment(),
+ * and reload hardware.
+ */
+void kvm_pmu_direct_swinc(struct kvm_vcpu *vcpu, u64 val)
+{
+ bool was_loaded;
+
+ guard(preempt)();
+
+ val &= kvm_vcpu_pmu_guest_counter_mask(vcpu) &
+ ~BIT_ULL(ARMV8_PMU_CYCLE_IDX);
+ if (!val)
+ return;
+
+ was_loaded = kvm_pmu_part_on_cpu(vcpu);
+ if (was_loaded)
+ __kvm_pmu_put(vcpu, false);
+
+ kvm_pmu_software_increment(vcpu, val);
+
+ if (was_loaded)
+ __kvm_pmu_load(vcpu, false);
+
+ if (kvm_pmu_part_overflow_status(vcpu))
+ kvm_make_request(KVM_REQ_IRQ_PENDING, vcpu);
+}
+
/**
* kvm_pmu_reload_part() - Reload partitioned PMU traps, filters, and state
* @vcpu: Pointer to struct kvm_vcpu
diff --git a/arch/arm64/kvm/pmu-emul.c b/arch/arm64/kvm/pmu-emul.c
index a72bdc5a0b3ea..b9096e5a4f11d 100644
--- a/arch/arm64/kvm/pmu-emul.c
+++ b/arch/arm64/kvm/pmu-emul.c
@@ -19,7 +19,6 @@
static void kvm_pmu_create_perf_event(struct kvm_pmc *pmc);
static void kvm_pmu_release_perf_event(struct kvm_pmc *pmc);
-static bool kvm_pmu_counter_is_enabled(struct kvm_pmc *pmc);
static struct kvm_vcpu *kvm_pmc_to_vcpu(const struct kvm_pmc *pmc)
{
@@ -48,6 +47,9 @@ static bool kvm_pmc_has_64bit_overflow(struct kvm_pmc *pmc)
struct kvm_vcpu *vcpu = kvm_pmc_to_vcpu(pmc);
u64 val = kvm_vcpu_read_pmcr(vcpu);
+ if (!kvm_pmc_is_64bit(pmc))
+ return false;
+
if (kvm_pmu_counter_is_hyp(vcpu, pmc->idx))
return __vcpu_sys_reg(vcpu, MDCR_EL2) & MDCR_EL2_HLP;
@@ -57,8 +59,13 @@ static bool kvm_pmc_has_64bit_overflow(struct kvm_pmc *pmc)
static bool kvm_pmu_counter_can_chain(struct kvm_pmc *pmc)
{
- return (!(pmc->idx & 1) && (pmc->idx + 1) < ARMV8_PMU_CYCLE_IDX &&
- !kvm_pmc_has_64bit_overflow(pmc));
+ struct kvm_vcpu *vcpu = kvm_pmc_to_vcpu(pmc);
+
+ return !(pmc->idx & 1) &&
+ (pmc->idx + 1) < vcpu->kvm->arch.nr_pmu_counters &&
+ kvm_pmu_counter_is_hyp(vcpu, pmc->idx) ==
+ kvm_pmu_counter_is_hyp(vcpu, pmc->idx + 1) &&
+ !kvm_pmc_has_64bit_overflow(pmc);
}
static u32 counter_index_to_reg(u64 idx)
@@ -76,6 +83,60 @@ static u64 kvm_pmc_read_evtreg(const struct kvm_pmc *pmc)
return __vcpu_sys_reg(kvm_pmc_to_vcpu(pmc), counter_index_to_evtreg(pmc->idx));
}
+static bool kvm_pmu_counter_is_enabled(struct kvm_pmc *pmc)
+{
+ struct kvm_vcpu *vcpu = kvm_pmc_to_vcpu(pmc);
+ unsigned int mdcr = __vcpu_sys_reg(vcpu, MDCR_EL2);
+
+ if (!(__vcpu_sys_reg(vcpu, PMCNTENSET_EL0) & BIT(pmc->idx)))
+ return false;
+
+ if (kvm_pmu_counter_is_hyp(vcpu, pmc->idx))
+ return mdcr & MDCR_EL2_HPME;
+
+ return kvm_vcpu_read_pmcr(vcpu) & ARMV8_PMU_PMCR_E;
+}
+
+static bool kvm_pmc_counts_at_el0(struct kvm_pmc *pmc)
+{
+ u64 evtreg = kvm_pmc_read_evtreg(pmc);
+ bool nsu = evtreg & ARMV8_PMU_EXCLUDE_NS_EL0;
+ bool u = evtreg & ARMV8_PMU_EXCLUDE_EL0;
+
+ return u == nsu;
+}
+
+static bool kvm_pmc_counts_at_el1(struct kvm_pmc *pmc)
+{
+ u64 evtreg = kvm_pmc_read_evtreg(pmc);
+ bool nsk = evtreg & ARMV8_PMU_EXCLUDE_NS_EL1;
+ bool p = evtreg & ARMV8_PMU_EXCLUDE_EL1;
+
+ return p == nsk;
+}
+
+static bool kvm_pmc_counts_at_el2(struct kvm_pmc *pmc)
+{
+ struct kvm_vcpu *vcpu = kvm_pmc_to_vcpu(pmc);
+ u64 mdcr = __vcpu_sys_reg(vcpu, MDCR_EL2);
+
+ if (!kvm_pmu_counter_is_hyp(vcpu, pmc->idx) && (mdcr & MDCR_EL2_HPMD))
+ return false;
+
+ return kvm_pmc_read_evtreg(pmc) & ARMV8_PMU_INCLUDE_EL2;
+}
+
+static bool kvm_pmc_counts_at_current_el(struct kvm_pmc *pmc)
+{
+ struct kvm_vcpu *vcpu = kvm_pmc_to_vcpu(pmc);
+
+ if (!vcpu_mode_priv(vcpu))
+ return kvm_pmc_counts_at_el0(pmc);
+ if (unlikely(is_hyp_ctxt(vcpu)))
+ return kvm_pmc_counts_at_el2(pmc);
+ return kvm_pmc_counts_at_el1(pmc);
+}
+
static u64 kvm_pmu_get_pmc_value(struct kvm_pmc *pmc)
{
struct kvm_vcpu *vcpu = kvm_pmc_to_vcpu(pmc);
@@ -321,9 +382,6 @@ static void kvm_pmu_counter_increment(struct kvm_vcpu *vcpu,
{
int i;
- if (!(kvm_vcpu_read_pmcr(vcpu) & ARMV8_PMU_PMCR_E))
- return;
-
/* Weed out disabled counters */
mask &= __vcpu_sys_reg(vcpu, PMCNTENSET_EL0);
@@ -331,12 +389,19 @@ static void kvm_pmu_counter_increment(struct kvm_vcpu *vcpu,
struct kvm_pmc *pmc = kvm_vcpu_idx_to_pmc(vcpu, i);
u64 type, reg;
+ if (!kvm_pmu_counter_is_enabled(pmc))
+ continue;
+
/* Filter on event type */
type = __vcpu_sys_reg(vcpu, counter_index_to_evtreg(i));
type &= kvm_pmu_event_mask(vcpu->kvm);
if (type != event)
continue;
+ if (event == ARMV8_PMUV3_PERFCTR_SW_INCR &&
+ !kvm_pmc_counts_at_current_el(pmc))
+ continue;
+
/* Increment this counter */
reg = __vcpu_sys_reg(vcpu, counter_index_to_reg(i)) + 1;
if (!kvm_pmc_is_64bit(pmc))
@@ -454,49 +519,6 @@ void kvm_pmu_handle_pmcr(struct kvm_vcpu *vcpu, u64 val)
}
}
-static bool kvm_pmu_counter_is_enabled(struct kvm_pmc *pmc)
-{
- struct kvm_vcpu *vcpu = kvm_pmc_to_vcpu(pmc);
- unsigned int mdcr = __vcpu_sys_reg(vcpu, MDCR_EL2);
-
- if (!(__vcpu_sys_reg(vcpu, PMCNTENSET_EL0) & BIT(pmc->idx)))
- return false;
-
- if (kvm_pmu_counter_is_hyp(vcpu, pmc->idx))
- return mdcr & MDCR_EL2_HPME;
-
- return kvm_vcpu_read_pmcr(vcpu) & ARMV8_PMU_PMCR_E;
-}
-
-static bool kvm_pmc_counts_at_el0(struct kvm_pmc *pmc)
-{
- u64 evtreg = kvm_pmc_read_evtreg(pmc);
- bool nsu = evtreg & ARMV8_PMU_EXCLUDE_NS_EL0;
- bool u = evtreg & ARMV8_PMU_EXCLUDE_EL0;
-
- return u == nsu;
-}
-
-static bool kvm_pmc_counts_at_el1(struct kvm_pmc *pmc)
-{
- u64 evtreg = kvm_pmc_read_evtreg(pmc);
- bool nsk = evtreg & ARMV8_PMU_EXCLUDE_NS_EL1;
- bool p = evtreg & ARMV8_PMU_EXCLUDE_EL1;
-
- return p == nsk;
-}
-
-static bool kvm_pmc_counts_at_el2(struct kvm_pmc *pmc)
-{
- struct kvm_vcpu *vcpu = kvm_pmc_to_vcpu(pmc);
- u64 mdcr = __vcpu_sys_reg(vcpu, MDCR_EL2);
-
- if (!kvm_pmu_counter_is_hyp(vcpu, pmc->idx) && (mdcr & MDCR_EL2_HPMD))
- return false;
-
- return kvm_pmc_read_evtreg(pmc) & ARMV8_PMU_INCLUDE_EL2;
-}
-
static int kvm_map_pmu_event(struct kvm *kvm, unsigned int eventsel)
{
struct arm_pmu *pmu = kvm->arch.arm_pmu;
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 24dd878fb58c4..4967954f42b01 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1543,7 +1543,10 @@ static bool access_pmswinc(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
return false;
mask = kvm_pmu_accessible_counter_mask(vcpu);
- kvm_pmu_software_increment(vcpu, p->regval & mask);
+ if (kvm_pmu_is_partitioned(vcpu->kvm))
+ kvm_pmu_direct_swinc(vcpu, p->regval & mask);
+ else
+ kvm_pmu_software_increment(vcpu, p->regval & mask);
return true;
}
diff --git a/include/kvm/arm_pmu.h b/include/kvm/arm_pmu.h
index 13e9230c9c714..0fe4f593d3f9a 100644
--- a/include/kvm/arm_pmu.h
+++ b/include/kvm/arm_pmu.h
@@ -110,6 +110,7 @@ u64 kvm_pmu_direct_reg_read(struct kvm_vcpu *vcpu, unsigned int reg);
void kvm_pmu_direct_reg_write(struct kvm_vcpu *vcpu, unsigned int reg,
u64 val, bool set);
u64 kvm_pmu_part_ovf32_mask(struct kvm_vcpu *vcpu);
+void kvm_pmu_direct_swinc(struct kvm_vcpu *vcpu, u64 val);
void kvm_pmu_reload_part(struct kvm_vcpu *vcpu);
u64 kvm_pmu_guest_counter_mask(void);
void kvm_pmu_load(struct kvm_vcpu *vcpu);
@@ -179,6 +180,8 @@ static inline u64 kvm_pmu_part_ovf32_mask(struct kvm_vcpu *vcpu)
{
return 0;
}
+
+static inline void kvm_pmu_direct_swinc(struct kvm_vcpu *vcpu, u64 val) {}
static inline void kvm_pmu_reload_part(struct kvm_vcpu *vcpu) {}
static inline void kvm_pmu_load(struct kvm_vcpu *vcpu) {}
static inline void kvm_pmu_put(struct kvm_vcpu *vcpu) {}
--
2.56.1.708.g77570a6a1d-goog
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v10 29/33] KVM: arm64: Forward FGT traps for PMXEVTYPER_EL0 and PMXEVCNTR_EL0
2026-11-19 18:51 [PATCH v10 00/33] ARM64 PMU Partitioning Colton Lewis
` (27 preceding siblings ...)
2026-11-19 18:52 ` [PATCH v10 28/33] KVM: arm64: Emulate trapped PMSWINC_EL0 writes for Partitioned PMU Colton Lewis
@ 2026-11-19 18:52 ` Colton Lewis
2026-11-19 19:15 ` sashiko-bot
2026-11-19 18:52 ` [PATCH v10 30/33] KVM: arm64: Support zero event counters on Partitioned PMU without HPMN0 Colton Lewis
` (3 subsequent siblings)
32 siblings, 1 reply; 67+ messages in thread
From: Colton Lewis @ 2026-11-19 18:52 UTC (permalink / raw)
To: Marc Zyngier, Oliver Upton
Cc: Catalin Marinas, Will Deacon, Mark Rutland, Peter Zijlstra,
Ingo Molnar, Paolo Bonzini, Russell King, Jonathan Corbet,
Fuad Tabba, Joey Gouly, Suzuki K Poulose, Zenghui Yu, Shuah Khan,
James Clark, Alexandru Elisei, Mingwei Zhang, Ganapatrao Kulkarni,
kvmarm, linux-arm-kernel, kvm, linux-perf-users, linux-kselftest,
linux-doc, linux-kernel
When an L1 hypervisor runs a nested L2 guest with a Partitioned PMU
(vMDCR_EL2.TPM == 0), L1 relies on fine-grained traps in HDFGRTR_EL2 and
HDFGWTR_EL2 (PMEVTYPERn_EL0, PMCCFILTR_EL0, and PMEVCNTRn_EL0) rather
than coarse-grained MDCR_EL2.TPM traps.
Add SYS_PMXEVTYPER_EL0 and SYS_PMXEVCNTR_EL0 to encoding_to_fgt[] in
arch/arm64/kvm/emulate-nested.c so L0 forwards fine-grained traps on
selector-indexed PMU accesses to L1, using PMSELR_SEL to check
HDFG{R,W}TR_EL2.PMCCFILTR_EL0 when PMSELR_EL0.SEL == 31 and
HDFG{R,W}TR_EL2.PMEVTYPERn_EL0 otherwise.
Signed-off-by: Colton Lewis <coltonlewis@google.com>
---
arch/arm64/kvm/emulate-nested.c | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/arch/arm64/kvm/emulate-nested.c b/arch/arm64/kvm/emulate-nested.c
index 1cd2f42452fe7..9da7e98b4d18f 100644
--- a/arch/arm64/kvm/emulate-nested.c
+++ b/arch/arm64/kvm/emulate-nested.c
@@ -1319,6 +1319,7 @@ static DEFINE_XARRAY(sr_forward_xa);
enum fg_filter_id {
__NO_FGF__,
HCRX_FGTnXS,
+ PMSELR_SEL,
/* Must be last */
__NR_FG_FILTER_IDS__
@@ -1870,6 +1871,8 @@ static const struct encoding_to_trap_config encoding_to_fgt[] __initconst = {
SR_FGT(SYS_PMCNTENSET_EL0, HDFGRTR, PMCNTEN, 1),
SR_FGT(SYS_PMCCNTR_EL0, HDFGRTR, PMCCNTR_EL0, 1),
SR_FGT(SYS_PMCCFILTR_EL0, HDFGRTR, PMCCFILTR_EL0, 1),
+ SR_FGF(SYS_PMXEVTYPER_EL0, HDFGRTR, PMEVTYPERn_EL0, 1, PMSELR_SEL),
+ SR_FGT(SYS_PMXEVCNTR_EL0, HDFGRTR, PMEVCNTRn_EL0, 1),
SR_FGT_RANGE(SYS_PMEVTYPERn_EL0(0),
SYS_PMEVTYPERn_EL0(30),
HDFGRTR, PMEVTYPERn_EL0, 1),
@@ -2609,6 +2612,16 @@ bool triage_sysreg_trap(struct kvm_vcpu *vcpu, int *sr_index)
case HDFGRTR_GROUP:
fgtreg = is_read ? HDFGRTR_EL2 : HDFGWTR_EL2;
+ switch (tc.fgf) {
+ case __NO_FGF__:
+ break;
+
+ case PMSELR_SEL:
+ if (SYS_FIELD_GET(PMSELR_EL0, SEL,
+ kvm_pmu_get_pmselr(vcpu)) == 31)
+ tc.bit = HDFGRTR_EL2_PMCCFILTR_EL0_SHIFT;
+ break;
+ }
break;
case HAFGRTR_GROUP:
--
2.56.1.708.g77570a6a1d-goog
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v10 30/33] KVM: arm64: Support zero event counters on Partitioned PMU without HPMN0
2026-11-19 18:51 [PATCH v10 00/33] ARM64 PMU Partitioning Colton Lewis
` (28 preceding siblings ...)
2026-11-19 18:52 ` [PATCH v10 29/33] KVM: arm64: Forward FGT traps for PMXEVTYPER_EL0 and PMXEVCNTR_EL0 Colton Lewis
@ 2026-11-19 18:52 ` Colton Lewis
2026-11-19 19:20 ` sashiko-bot
2026-11-19 18:52 ` [PATCH v10 31/33] KVM: arm64: Add vCPU device attr to partition the PMU Colton Lewis
` (2 subsequent siblings)
32 siblings, 1 reply; 67+ messages in thread
From: Colton Lewis @ 2026-11-19 18:52 UTC (permalink / raw)
To: Marc Zyngier, Oliver Upton
Cc: Catalin Marinas, Will Deacon, Mark Rutland, Peter Zijlstra,
Ingo Molnar, Paolo Bonzini, Russell King, Jonathan Corbet,
Fuad Tabba, Joey Gouly, Suzuki K Poulose, Zenghui Yu, Shuah Khan,
James Clark, Alexandru Elisei, Mingwei Zhang, Ganapatrao Kulkarni,
kvmarm, linux-arm-kernel, kvm, linux-perf-users, linux-kselftest,
linux-doc, linux-kernel
Programming MDCR_EL2.HPMN to 0 is CONSTRAINED UNPREDICTABLE on CPUs that
do not implement FEAT_HPMN0, preventing userspace from configuring a
cycle-counter-only Partitioned PMU guest (nr_pmu_counters == 0) on those
platforms if HPMN is set directly from nr_pmu_counters.
Support nr_pmu_counters == 0 without FEAT_HPMN0 whenever the PMU has at
least two general-purpose counters by setting physical MDCR_EL2.HPMN = 1
(via kvm_arm_pmu_part_hpmn()), reserving and quiescing physical counter 0
alongside PMCCNTR_EL0 on vCPU load so host perf uses counters 1..N-1,
keeping MDCR_EL2.TPMCR set so trapped PMCR_EL0 reads report N == 0, and
enabling fine-grained traps on PMEVCNTRn_EL0, PMCNTEN, PMINTEN, and
PMSWINC_EL0 so counter 0 remains completely hidden from the guest while
PMCCNTR_EL0 accesses stay untrapped on FEAT_FGT hosts.
Signed-off-by: Colton Lewis <coltonlewis@google.com>
---
arch/arm64/kvm/config.c | 16 ++++++++++----
arch/arm64/kvm/debug.c | 12 ++++++-----
arch/arm64/kvm/pmu-direct.c | 42 ++++++++++++++++++++++++++++++-------
arch/arm64/kvm/pmu.c | 15 +++++++++++--
include/kvm/arm_pmu.h | 6 ++++++
5 files changed, 73 insertions(+), 18 deletions(-)
diff --git a/arch/arm64/kvm/config.c b/arch/arm64/kvm/config.c
index 0ec74e45fdbf6..bbe940f6266aa 100644
--- a/arch/arm64/kvm/config.c
+++ b/arch/arm64/kvm/config.c
@@ -1791,6 +1791,8 @@ static void __compute_hdfgrtr(struct kvm_vcpu *vcpu)
__compute_fgt(vcpu, HDFGRTR_EL2);
if (kvm_pmu_is_partitioned(vcpu->kvm)) {
+ unsigned int n = vcpu->kvm->arch.nr_pmu_counters;
+ bool hide_gp0 = kvm_arm_pmu_part_hpmn(n) != n;
u64 set = HDFGRTR_EL2_PMOVS |
HDFGRTR_EL2_PMCCFILTR_EL0 |
HDFGRTR_EL2_PMEVTYPERn_EL0 |
@@ -1801,10 +1803,12 @@ static void __compute_hdfgrtr(struct kvm_vcpu *vcpu)
(__vcpu_sys_reg(vcpu, MDCR_EL2) & MDCR_EL2_TPM))
set |= HDFGRTR_EL2_PMUSERENR_EL0;
- if (kvm_pmu_hyp_counter_mask(vcpu))
+ if (kvm_pmu_hyp_counter_mask(vcpu) || hide_gp0)
set |= HDFGRTR_EL2_PMCNTEN;
- if (kvm_pmu_part_ovf32_mask(vcpu))
+ if (kvm_pmu_part_ovf32_mask(vcpu) || hide_gp0)
set |= HDFGRTR_EL2_PMEVCNTRn_EL0;
+ if (hide_gp0)
+ set |= HDFGRTR_EL2_PMINTEN;
*vcpu_fgt(vcpu, HDFGRTR_EL2) |= set & ~hdfgrtr_masks.res0;
}
@@ -1818,6 +1822,8 @@ static void __compute_hdfgwtr(struct kvm_vcpu *vcpu)
*vcpu_fgt(vcpu, HDFGWTR_EL2) |= HDFGWTR_EL2_MDSCR_EL1;
if (kvm_pmu_is_partitioned(vcpu->kvm)) {
+ unsigned int n = vcpu->kvm->arch.nr_pmu_counters;
+ bool hide_gp0 = kvm_arm_pmu_part_hpmn(n) != n;
u64 set = HDFGWTR_EL2_PMOVS |
HDFGWTR_EL2_PMCCFILTR_EL0 |
HDFGWTR_EL2_PMEVTYPERn_EL0;
@@ -1825,10 +1831,12 @@ static void __compute_hdfgwtr(struct kvm_vcpu *vcpu)
if (is_hyp_ctxt(vcpu))
set |= HDFGWTR_EL2_PMUSERENR_EL0;
- if (kvm_pmu_hyp_counter_mask(vcpu))
+ if (kvm_pmu_hyp_counter_mask(vcpu) || hide_gp0)
set |= HDFGWTR_EL2_PMCNTEN | HDFGWTR_EL2_PMSWINC_EL0;
- if (kvm_pmu_part_ovf32_mask(vcpu))
+ if (kvm_pmu_part_ovf32_mask(vcpu) || hide_gp0)
set |= HDFGWTR_EL2_PMEVCNTRn_EL0;
+ if (hide_gp0)
+ set |= HDFGWTR_EL2_PMINTEN;
*vcpu_fgt(vcpu, HDFGWTR_EL2) |= set & ~hdfgwtr_masks.res0;
}
diff --git a/arch/arm64/kvm/debug.c b/arch/arm64/kvm/debug.c
index b89f46cb59955..73d157f5b8f6f 100644
--- a/arch/arm64/kvm/debug.c
+++ b/arch/arm64/kvm/debug.c
@@ -58,11 +58,11 @@ void kvm_arm_setup_mdcr_el2(struct kvm_vcpu *vcpu)
if (kvm_pmu_is_partitioned(vcpu->kvm) && vcpu->arch.pmu.loaded_on_cpu) {
u8 nr_host_cntr = *host_data_ptr(nr_event_counters);
u8 nr_guest_cntr = vcpu->kvm->arch.nr_pmu_counters;
+ u8 guest_hpmn = kvm_arm_pmu_part_hpmn(nr_guest_cntr);
if (cpumask_test_cpu(smp_processor_id(),
vcpu->kvm->arch.supported_cpus) &&
- nr_guest_cntr < nr_host_cntr &&
- (cpus_have_final_cap(ARM64_HAS_HPMN0) || nr_guest_cntr > 0)) {
+ guest_hpmn < nr_host_cntr) {
u64 hpmn = FIELD_GET(MDCR_EL2_HPMN, read_sysreg(mdcr_el2));
bool hpme;
@@ -81,14 +81,16 @@ void kvm_arm_setup_mdcr_el2(struct kvm_vcpu *vcpu)
* Take out the coarse grain traps if we are using
* fine grain traps and enforce counter access with
* HPMN. Keep TPMCR when L1 sub-partitions its counters
- * so PMCR_EL0 reads and writes trap for multiplexing.
+ * or when guest_hpmn > nr_guest_cntr (!FEAT_HPMN0 with
+ * 0 event counters) so PMCR_EL0 accesses trap.
*/
mdcr &= ~MDCR_EL2_HPMN;
- mdcr |= FIELD_PREP(MDCR_EL2_HPMN, nr_guest_cntr);
+ mdcr |= FIELD_PREP(MDCR_EL2_HPMN, guest_hpmn);
if (cpus_have_final_cap(ARM64_HAS_FGT)) {
mdcr &= ~MDCR_EL2_TPM;
- if (!kvm_pmu_hyp_counter_mask(vcpu))
+ if (!kvm_pmu_hyp_counter_mask(vcpu) &&
+ guest_hpmn == nr_guest_cntr)
mdcr &= ~MDCR_EL2_TPMCR;
}
}
diff --git a/arch/arm64/kvm/pmu-direct.c b/arch/arm64/kvm/pmu-direct.c
index 6382233e54c71..7b8f0a69641bc 100644
--- a/arch/arm64/kvm/pmu-direct.c
+++ b/arch/arm64/kvm/pmu-direct.c
@@ -89,6 +89,22 @@ static u64 kvm_vcpu_pmu_guest_counter_mask(struct kvm_vcpu *vcpu)
return 0;
}
+static u64 kvm_vcpu_pmu_reserved_counter_mask(struct kvm_vcpu *vcpu)
+{
+ if (vcpu && kvm_pmu_is_partitioned(vcpu->kvm)) {
+ u64 mask = ARMV8_PMU_CNT_MASK_C;
+ unsigned int hpmn =
+ kvm_arm_pmu_part_hpmn(vcpu->kvm->arch.nr_pmu_counters);
+
+ if (hpmn > 0)
+ mask |= GENMASK_ULL(hpmn - 1, 0);
+
+ return mask;
+ }
+
+ return 0;
+}
+
static void kvm_pmu_set_guest_owned(struct kvm_vcpu *vcpu);
/**
@@ -372,8 +388,12 @@ static u64 kvm_pmu_direct_pmcr_read(struct kvm_vcpu *vcpu)
if (!kvm_has_feat(vcpu->kvm, ID_AA64DFR0_EL1, PMUVer, V3P5))
pmcr &= ~ARMV8_PMU_PMCR_LP;
- if (vcpu_has_nv(vcpu) && !vcpu_is_el2(vcpu))
- n = FIELD_GET(MDCR_EL2_HPMN, __vcpu_sys_reg(vcpu, MDCR_EL2));
+ if (vcpu_has_nv(vcpu) && !vcpu_is_el2(vcpu)) {
+ u64 hpmn = FIELD_GET(MDCR_EL2_HPMN, __vcpu_sys_reg(vcpu, MDCR_EL2));
+
+ if (hpmn < n)
+ n = hpmn;
+ }
return u64_replace_bits(pmcr, n, ARMV8_PMU_PMCR_N);
}
@@ -686,6 +706,7 @@ static void kvm_pmu_apply_event_filter(struct kvm_vcpu *vcpu)
static void __kvm_pmu_load(struct kvm_vcpu *vcpu, bool reserve_counters)
{
struct arm_pmu *pmu = vcpu->kvm->arch.arm_pmu;
+ unsigned long reserved_counters;
unsigned long guest_counters;
unsigned long flags;
unsigned long daif;
@@ -710,20 +731,27 @@ static void __kvm_pmu_load(struct kvm_vcpu *vcpu, bool reserve_counters)
return;
}
+ reserved_counters = kvm_vcpu_pmu_reserved_counter_mask(vcpu);
guest_counters = kvm_vcpu_pmu_guest_counter_mask(vcpu);
if (reserve_counters)
- kvm_pmu_set_guest_counters(pmu, guest_counters);
+ kvm_pmu_set_guest_counters(pmu, reserved_counters);
daif = local_daif_save();
/*
* Ensure guest counters, interrupts, and overflow flags are quiesced
* in hardware before lowering MDCR_EL2.HPMN and marking the partition
- * live on this CPU.
+ * live on this CPU. If counter 0 is reserved solely to satisfy HPMN=1
+ * without FEAT_HPMN0, scrub and filter it so it cannot count or leak.
*/
- write_sysreg(guest_counters, pmcntenclr_el0);
- write_sysreg(guest_counters, pmintenclr_el1);
- write_sysreg(guest_counters, pmovsclr_el0);
+ write_sysreg(reserved_counters, pmcntenclr_el0);
+ write_sysreg(reserved_counters, pmintenclr_el1);
+ write_sysreg(reserved_counters, pmovsclr_el0);
+ if (reserved_counters & ~guest_counters) {
+ write_pmevcntrn(0, 0);
+ write_pmevtypern(0, ARMV8_PMU_EXCLUDE_EL0 |
+ ARMV8_PMU_EXCLUDE_EL1);
+ }
isb();
vcpu->arch.pmu.loaded_on_cpu = true;
diff --git a/arch/arm64/kvm/pmu.c b/arch/arm64/kvm/pmu.c
index 9d4ad4b7bd8d5..02a1968bd6cb1 100644
--- a/arch/arm64/kvm/pmu.c
+++ b/arch/arm64/kvm/pmu.c
@@ -914,6 +914,13 @@ u64 kvm_pmu_accessible_counter_mask(struct kvm_vcpu *vcpu)
return mask & ~kvm_pmu_hyp_counter_mask(vcpu);
}
+unsigned int kvm_arm_pmu_part_hpmn(unsigned int n)
+{
+ if (!n && !cpus_have_final_cap(ARM64_HAS_HPMN0))
+ return 1;
+ return n;
+}
+
/**
* kvm_vcpu_read_pmcr - Read PMCR_EL0 register for the vCPU
* @vcpu: The vcpu pointer
@@ -923,8 +930,12 @@ u64 kvm_vcpu_read_pmcr(struct kvm_vcpu *vcpu)
u64 pmcr = __vcpu_sys_reg(vcpu, PMCR_EL0);
u64 n = vcpu->kvm->arch.nr_pmu_counters;
- if (vcpu_has_nv(vcpu) && !vcpu_is_el2(vcpu))
- n = FIELD_GET(MDCR_EL2_HPMN, __vcpu_sys_reg(vcpu, MDCR_EL2));
+ if (vcpu_has_nv(vcpu) && !vcpu_is_el2(vcpu)) {
+ u64 hpmn = FIELD_GET(MDCR_EL2_HPMN, __vcpu_sys_reg(vcpu, MDCR_EL2));
+
+ if (hpmn < n)
+ n = hpmn;
+ }
return u64_replace_bits(pmcr, n, ARMV8_PMU_PMCR_N);
}
diff --git a/include/kvm/arm_pmu.h b/include/kvm/arm_pmu.h
index 0fe4f593d3f9a..bebf981855040 100644
--- a/include/kvm/arm_pmu.h
+++ b/include/kvm/arm_pmu.h
@@ -137,6 +137,7 @@ u8 kvm_arm_pmu_get_pmuver_limit(void);
u64 kvm_pmu_evtyper_mask(struct kvm *kvm);
int kvm_arm_set_default_pmu(struct kvm *kvm);
u8 kvm_arm_pmu_get_max_counters(struct kvm *kvm);
+unsigned int kvm_arm_pmu_part_hpmn(unsigned int n);
u64 kvm_vcpu_read_pmcr(struct kvm_vcpu *vcpu);
bool kvm_pmu_counter_is_hyp(struct kvm_vcpu *vcpu, unsigned int idx);
@@ -277,6 +278,11 @@ static inline u8 kvm_arm_pmu_get_max_counters(struct kvm *kvm)
return 0;
}
+static inline unsigned int kvm_arm_pmu_part_hpmn(unsigned int n)
+{
+ return n;
+}
+
static inline u64 kvm_vcpu_read_pmcr(struct kvm_vcpu *vcpu)
{
return 0;
--
2.56.1.708.g77570a6a1d-goog
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v10 31/33] KVM: arm64: Add vCPU device attr to partition the PMU
2026-11-19 18:51 [PATCH v10 00/33] ARM64 PMU Partitioning Colton Lewis
` (29 preceding siblings ...)
2026-11-19 18:52 ` [PATCH v10 30/33] KVM: arm64: Support zero event counters on Partitioned PMU without HPMN0 Colton Lewis
@ 2026-11-19 18:52 ` Colton Lewis
2026-11-19 19:14 ` sashiko-bot
2026-11-19 18:52 ` [PATCH v10 32/33] KVM: selftests: Add find_bit to KVM library Colton Lewis
2026-11-19 18:52 ` [PATCH v10 33/33] KVM: arm64: selftests: Add test case for Partitioned PMU Colton Lewis
32 siblings, 1 reply; 67+ messages in thread
From: Colton Lewis @ 2026-11-19 18:52 UTC (permalink / raw)
To: Marc Zyngier, Oliver Upton
Cc: Catalin Marinas, Will Deacon, Mark Rutland, Peter Zijlstra,
Ingo Molnar, Paolo Bonzini, Russell King, Jonathan Corbet,
Fuad Tabba, Joey Gouly, Suzuki K Poulose, Zenghui Yu, Shuah Khan,
James Clark, Alexandru Elisei, Mingwei Zhang, Ganapatrao Kulkarni,
kvmarm, linux-arm-kernel, kvm, linux-perf-users, linux-kselftest,
linux-doc, linux-kernel
Add the KVM_ARM_VCPU_PMU_V3_ENABLE_PARTITION vCPU device attribute to
enable a Partitioned PMU on hosts with VHE and PMUv3 support.
When partitioning is enabled (KVM_ARCH_FLAG_PARTITION_PMU_ENABLED),
userspace must configure guest general-purpose counters via
KVM_ARM_VCPU_PMU_V3_SET_NR_COUNTERS before KVM_ARM_VCPU_PMU_V3_INIT:
- Require kvm_arm_pmu_part_hpmn(n) < max_counters so at least one
general-purpose counter remains available for host profiling.
- Allow n == 0 (cycle counter only) on hosts without FEAT_HPMN0 when
max_counters >= 2, reserving physical counter 0 (MDCR_EL2.HPMN = 1)
and hiding it from the guest via traps.
Because PMU attributes are invoked per-vCPU while arm_pmu,
nr_pmu_counters, and partitioning state are VM-wide, VMMs such as QEMU
iterate over vCPUs (CPU_FOREACH) and call KVM_ARM_VCPU_PMU_V3_INIT on
vCPU 0 before configuring vCPU 1..N. Allow SET_PMU, SET_NR_COUNTERS, and
ENABLE_PARTITION to succeed when a vCPU PMU is already initialized as
long as the requested value matches the VM's existing configuration, and
return -EBUSY on conflicting changes.
Signed-off-by: Colton Lewis <coltonlewis@google.com>
---
Documentation/virt/kvm/devices/vcpu.rst | 57 +++++++++++++++--
arch/arm64/include/asm/kvm_host.h | 1 +
arch/arm64/include/uapi/asm/kvm.h | 2 +
arch/arm64/kvm/pmu.c | 83 ++++++++++++++++++++++++-
arch/arm64/kvm/sys_regs.c | 5 +-
include/kvm/arm_pmu.h | 6 ++
6 files changed, 148 insertions(+), 6 deletions(-)
diff --git a/Documentation/virt/kvm/devices/vcpu.rst b/Documentation/virt/kvm/devices/vcpu.rst
index 7e23584fd74aa..1140c1d731d63 100644
--- a/Documentation/virt/kvm/devices/vcpu.rst
+++ b/Documentation/virt/kvm/devices/vcpu.rst
@@ -57,6 +57,9 @@ Returns:
hardware PMU, or interrupt number not set (non-GICv5
guests, only)
-EBUSY PMUv3 already initialized
+ -EINVAL Partitioning enabled without explicitly configuring
+ a valid counter count via
+ KVM_ARM_VCPU_PMU_V3_SET_NR_COUNTERS
======= ======================================================
Request the initialization of the PMUv3. If using the PMUv3 with an in-kernel
@@ -161,16 +164,62 @@ the cpu field to the processor id.
an event filter has already been set
-EFAULT Error accessing the value pointed to by addr
-ENODEV PMUv3 not supported or GIC not initialized
- -EINVAL No PMUv3 explicitly selected, or value of N out of
- range
+ -EINVAL No PMUv3 explicitly selected, or value of N > max_counters,
+ or N > max_counters - 1 (or N == 0 when max_counters < 2
+ without FEAT_HPMN0) when partitioning is enabled
======= ====================================================
Set the number of implemented event counters in the virtual PMU. This
mandates that a PMU has explicitly been selected via
KVM_ARM_VCPU_PMU_V3_SET_PMU, and will fail when no PMU has been
explicitly selected, or the number of counters is out of range for the
-selected PMU. Selecting a new PMU cancels the effect of setting this
-attribute.
+selected PMU. The maximum number of general-purpose counters (max_counters)
+implemented by a PMU can be queried from sysfs at
+/sys/bus/event_source/devices/<pmu>/caps/num_counters. Selecting a new PMU
+cancels the effect of setting this attribute.
+
+1.6 ATTRIBUTE: KVM_ARM_VCPU_PMU_V3_ENABLE_PARTITION
+---------------------------------------------------
+
+:Parameters: in kvm_device_attr.addr the address to an unsigned int (u32)
+ boolean value (non-zero to enable PMU partitioning, 0 to disable)
+
+:Returns:
+
+ ======= ========================================================
+ -EBUSY PMUv3 already initialized or a VCPU has already run
+ -EFAULT Error accessing the value pointed to by addr
+ -ENODEV KVM_ARM_VCPU_PMU_V3 feature missing from VCPU
+ -EPERM Host hardware or kernel configuration does not support
+ PMU partitioning (requires ARM64 VHE mode and PMUv3)
+ -EINVAL No PMUv3 associated with the VM, or
+ KVM_ARM_VCPU_PMU_V3_SET_NR_COUNTERS was already set to
+ > max_counters - 1 (or 0 when max_counters < 2 without
+ FEAT_HPMN0)
+ -ENXIO Returned by KVM_HAS_DEVICE_ATTR when PMU partitioning is
+ unsupported on host hardware
+ ======= ========================================================
+
+Enable or disable hardware PMU partitioning for the VM. When enabled, physical
+PMUv3 hardware counters are partitioned between the guest and host using
+MDCR_EL2.HPMN (and FEAT_FGT fine-grained traps when supported by hardware).
+This grants the guest direct, untrapped EL0/EL1 hardware access to event
+counters 0..HPMN-1 and the cycle counter (PMCCNTR_EL0), and restricts the VM's
+supported physical CPUs to those associated with the VM's PMU.
+
+When PMU partitioning is enabled, userspace must explicitly configure the
+number of guest event counters via KVM_ARM_VCPU_PMU_V3_SET_NR_COUNTERS with a
+value strictly less than the maximum number of general-purpose counters
+implemented by the PMU (leaving at least one general-purpose counter reserved
+for host profiling) prior to calling KVM_ARM_VCPU_PMU_V3_INIT. Configuring 0
+event counters (cycle counter only) on hosts without FEAT_HPMN0 programs
+MDCR_EL2.HPMN = 1, reserves physical counter 0 in addition to PMCCNTR_EL0
+while hiding it from the guest via traps, and therefore requires at least two
+implemented general-purpose counters (max_counters >= 2) so at least one
+remains for host profiling. Note that KVM_ARM_VCPU_PMU_V3_SET_NR_COUNTERS
+configures general-purpose event counters (PMCR_EL0.N / MDCR_EL2.HPMN); the
+dedicated cycle counter (PMCCNTR_EL0) is unconditionally assigned to the
+guest partition when partitioning is enabled.
2. GROUP: KVM_ARM_VCPU_TIMER_CTRL
=================================
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 1fcb12995c34c..01f26a37c7f9c 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -472,6 +472,7 @@ struct kvm_arch {
/* Maximum number of counters for the guest */
u8 nr_pmu_counters;
+ bool pmu_nr_counters_specified;
/* PMMIR_EL1.SLOTS value exposed to the guest. */
u8 pmmir_slots;
diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
index 65ee09bd14636..80e2504c1d9fb 100644
--- a/arch/arm64/include/uapi/asm/kvm.h
+++ b/arch/arm64/include/uapi/asm/kvm.h
@@ -453,6 +453,8 @@ struct kvm_vgic_v5_ist {
#define KVM_ARM_VCPU_PMU_V3_FILTER 2
#define KVM_ARM_VCPU_PMU_V3_SET_PMU 3
#define KVM_ARM_VCPU_PMU_V3_SET_NR_COUNTERS 4
+#define KVM_ARM_VCPU_PMU_V3_ENABLE_PARTITION 5
+
#define KVM_ARM_VCPU_TIMER_CTRL 1
#define KVM_ARM_VCPU_TIMER_IRQ_VTIMER 0
#define KVM_ARM_VCPU_TIMER_IRQ_PTIMER 1
diff --git a/arch/arm64/kvm/pmu.c b/arch/arm64/kvm/pmu.c
index 02a1968bd6cb1..e11f54a92ecfe 100644
--- a/arch/arm64/kvm/pmu.c
+++ b/arch/arm64/kvm/pmu.c
@@ -479,12 +479,22 @@ int kvm_arm_pmu_v3_enable(struct kvm_vcpu *vcpu)
return 0;
}
+bool kvm_arm_pmu_part_nr_valid(struct kvm *kvm, unsigned int n)
+{
+ return kvm_arm_pmu_part_hpmn(n) < kvm_arm_pmu_get_max_counters(kvm);
+}
+
static int kvm_arm_pmu_v3_init(struct kvm_vcpu *vcpu)
{
/* Only possible when using KVM_ARM_VCPU_PMU_V3_STRICT */
if (!vcpu->kvm->arch.arm_pmu)
return -ENXIO;
+ if (kvm_pmu_is_partitioned(vcpu->kvm) &&
+ (!vcpu->kvm->arch.pmu_nr_counters_specified ||
+ !kvm_arm_pmu_part_nr_valid(vcpu->kvm, vcpu->kvm->arch.nr_pmu_counters)))
+ return -EINVAL;
+
if (irqchip_in_kernel(vcpu->kvm)) {
int ret;
@@ -596,11 +606,25 @@ static void kvm_arm_set_nr_counters(struct kvm *kvm, unsigned int nr)
}
}
+static bool kvm_arm_pmu_any_vcpu_created(struct kvm *kvm)
+{
+ struct kvm_vcpu *vcpu;
+ unsigned long i;
+
+ kvm_for_each_vcpu(i, vcpu, kvm) {
+ if (vcpu->arch.pmu.created)
+ return true;
+ }
+
+ return false;
+}
+
static void kvm_arm_set_pmu(struct kvm *kvm, struct arm_pmu *arm_pmu)
{
lockdep_assert_held(&kvm->arch.config_lock);
kvm->arch.arm_pmu = arm_pmu;
+ kvm->arch.pmu_nr_counters_specified = false;
kvm_arm_set_nr_counters(kvm, kvm_arm_pmu_get_max_counters(kvm));
}
@@ -647,6 +671,11 @@ static int kvm_arm_pmu_v3_set_pmu(struct kvm_vcpu *vcpu, int pmu_id)
break;
}
+ if (kvm_arm_pmu_any_vcpu_created(kvm)) {
+ ret = (kvm->arch.arm_pmu == arm_pmu) ? 0 : -EBUSY;
+ break;
+ }
+
kvm_arm_set_pmu(kvm, arm_pmu);
cpumask_copy(kvm->arch.supported_cpus, &arm_pmu->supported_cpus);
@@ -672,14 +701,31 @@ static int kvm_arm_pmu_v3_set_pmu(struct kvm_vcpu *vcpu, int pmu_id)
static int kvm_arm_pmu_v3_set_nr_counters(struct kvm_vcpu *vcpu, unsigned int n)
{
struct kvm *kvm = vcpu->kvm;
+ unsigned int max_counters;
+
+ if (kvm_vm_has_ran_once(kvm))
+ return -EBUSY;
+
+ if (kvm_arm_pmu_any_vcpu_created(kvm)) {
+ if (!kvm->arch.pmu_nr_counters_specified ||
+ kvm->arch.nr_pmu_counters != n)
+ return -EBUSY;
+ return 0;
+ }
if (!kvm->arch.arm_pmu)
return -EINVAL;
- if (n > kvm_arm_pmu_get_max_counters(kvm))
+ max_counters = kvm_arm_pmu_get_max_counters(kvm);
+ if (n > max_counters)
+ return -EINVAL;
+
+ if (kvm_pmu_is_partitioned(kvm) &&
+ !kvm_arm_pmu_part_nr_valid(kvm, n))
return -EINVAL;
kvm_arm_set_nr_counters(kvm, n);
+ kvm->arch.pmu_nr_counters_specified = true;
return 0;
}
@@ -791,6 +837,36 @@ int kvm_arm_pmu_v3_set_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
return kvm_arm_pmu_v3_set_nr_counters(vcpu, n);
}
+ case KVM_ARM_VCPU_PMU_V3_ENABLE_PARTITION: {
+ unsigned int __user *uaddr = (unsigned int __user *)(long)attr->addr;
+ u32 val;
+
+ if (get_user(val, uaddr))
+ return -EFAULT;
+
+ if (!has_kvm_pmu_partition_support())
+ return -EPERM;
+
+ if (kvm_vm_has_ran_once(kvm))
+ return -EBUSY;
+
+ if (kvm_arm_pmu_any_vcpu_created(kvm))
+ return kvm_pmu_is_partitioned(kvm) == !!val ? 0 : -EBUSY;
+
+ if (!kvm->arch.arm_pmu)
+ return -EINVAL;
+
+ if (val && kvm->arch.pmu_nr_counters_specified &&
+ !kvm_arm_pmu_part_nr_valid(kvm, kvm->arch.nr_pmu_counters))
+ return -EINVAL;
+
+ if (val)
+ cpumask_copy(kvm->arch.supported_cpus,
+ &kvm->arch.arm_pmu->supported_cpus);
+
+ kvm_pmu_partition_enable(kvm, val);
+ return 0;
+ }
case KVM_ARM_VCPU_PMU_V3_INIT:
return kvm_arm_pmu_v3_init(vcpu);
}
@@ -832,6 +908,11 @@ int kvm_arm_pmu_v3_has_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
case KVM_ARM_VCPU_PMU_V3_SET_NR_COUNTERS:
if (kvm_vcpu_has_pmu(vcpu))
return 0;
+ break;
+ case KVM_ARM_VCPU_PMU_V3_ENABLE_PARTITION:
+ if (kvm_vcpu_has_pmu(vcpu) && has_kvm_pmu_partition_support())
+ return 0;
+ break;
}
return -ENXIO;
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 4967954f42b01..22d78605b8f65 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1591,7 +1591,10 @@ static int set_pmcr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r,
if (!kvm_vm_has_ran_once(kvm) &&
!vcpu_has_nv(vcpu) &&
!kvm_vcpu_has_pmuv3_strict(vcpu) &&
- new_n <= kvm_arm_pmu_get_max_counters(kvm))
+ !kvm->arch.pmu_nr_counters_specified &&
+ new_n <= kvm_arm_pmu_get_max_counters(kvm) &&
+ (!kvm_pmu_is_partitioned(kvm) ||
+ kvm_arm_pmu_part_nr_valid(kvm, new_n)))
kvm->arch.nr_pmu_counters = new_n;
mutex_unlock(&kvm->arch.config_lock);
diff --git a/include/kvm/arm_pmu.h b/include/kvm/arm_pmu.h
index bebf981855040..479eabc96b412 100644
--- a/include/kvm/arm_pmu.h
+++ b/include/kvm/arm_pmu.h
@@ -138,6 +138,7 @@ u64 kvm_pmu_evtyper_mask(struct kvm *kvm);
int kvm_arm_set_default_pmu(struct kvm *kvm);
u8 kvm_arm_pmu_get_max_counters(struct kvm *kvm);
unsigned int kvm_arm_pmu_part_hpmn(unsigned int n);
+bool kvm_arm_pmu_part_nr_valid(struct kvm *kvm, unsigned int n);
u64 kvm_vcpu_read_pmcr(struct kvm_vcpu *vcpu);
bool kvm_pmu_counter_is_hyp(struct kvm_vcpu *vcpu, unsigned int idx);
@@ -283,6 +284,11 @@ static inline unsigned int kvm_arm_pmu_part_hpmn(unsigned int n)
return n;
}
+static inline bool kvm_arm_pmu_part_nr_valid(struct kvm *kvm, unsigned int n)
+{
+ return false;
+}
+
static inline u64 kvm_vcpu_read_pmcr(struct kvm_vcpu *vcpu)
{
return 0;
--
2.56.1.708.g77570a6a1d-goog
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v10 32/33] KVM: selftests: Add find_bit to KVM library
2026-11-19 18:51 [PATCH v10 00/33] ARM64 PMU Partitioning Colton Lewis
` (30 preceding siblings ...)
2026-11-19 18:52 ` [PATCH v10 31/33] KVM: arm64: Add vCPU device attr to partition the PMU Colton Lewis
@ 2026-11-19 18:52 ` Colton Lewis
2026-11-19 19:07 ` sashiko-bot
2026-11-19 18:52 ` [PATCH v10 33/33] KVM: arm64: selftests: Add test case for Partitioned PMU Colton Lewis
32 siblings, 1 reply; 67+ messages in thread
From: Colton Lewis @ 2026-11-19 18:52 UTC (permalink / raw)
To: Marc Zyngier, Oliver Upton
Cc: Catalin Marinas, Will Deacon, Mark Rutland, Peter Zijlstra,
Ingo Molnar, Paolo Bonzini, Russell King, Jonathan Corbet,
Fuad Tabba, Joey Gouly, Suzuki K Poulose, Zenghui Yu, Shuah Khan,
James Clark, Alexandru Elisei, Mingwei Zhang, Ganapatrao Kulkarni,
kvmarm, linux-arm-kernel, kvm, linux-perf-users, linux-kselftest,
linux-doc, linux-kernel
Add lib/find_bit.c to the KVM selftests build using the same wrapper
pattern as lib/rbtree.c so selftests that depend on find_bit helpers
link cleanly when built standalone.
Tested-by: James Clark <james.clark@linaro.org>
Signed-off-by: Colton Lewis <coltonlewis@google.com>
---
tools/testing/selftests/kvm/Makefile.kvm | 1 +
tools/testing/selftests/kvm/lib/find_bit.c | 2 ++
2 files changed, 3 insertions(+)
create mode 100644 tools/testing/selftests/kvm/lib/find_bit.c
diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
index 17a4256e90819..79b67a86208c3 100644
--- a/tools/testing/selftests/kvm/Makefile.kvm
+++ b/tools/testing/selftests/kvm/Makefile.kvm
@@ -5,6 +5,7 @@ all:
LIBKVM += lib/assert.c
LIBKVM += lib/elf.c
+LIBKVM += lib/find_bit.c
LIBKVM += lib/guest_modes.c
LIBKVM += lib/kvm_util.c
LIBKVM += lib/lru_gen_util.c
diff --git a/tools/testing/selftests/kvm/lib/find_bit.c b/tools/testing/selftests/kvm/lib/find_bit.c
new file mode 100644
index 0000000000000..5534248c663f7
--- /dev/null
+++ b/tools/testing/selftests/kvm/lib/find_bit.c
@@ -0,0 +1,2 @@
+// SPDX-License-Identifier: GPL-2.0
+#include "../../../../lib/find_bit.c"
--
2.56.1.708.g77570a6a1d-goog
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v10 33/33] KVM: arm64: selftests: Add test case for Partitioned PMU
2026-11-19 18:51 [PATCH v10 00/33] ARM64 PMU Partitioning Colton Lewis
` (31 preceding siblings ...)
2026-11-19 18:52 ` [PATCH v10 32/33] KVM: selftests: Add find_bit to KVM library Colton Lewis
@ 2026-11-19 18:52 ` Colton Lewis
2026-11-19 19:14 ` sashiko-bot
32 siblings, 1 reply; 67+ messages in thread
From: Colton Lewis @ 2026-11-19 18:52 UTC (permalink / raw)
To: Marc Zyngier, Oliver Upton
Cc: Catalin Marinas, Will Deacon, Mark Rutland, Peter Zijlstra,
Ingo Molnar, Paolo Bonzini, Russell King, Jonathan Corbet,
Fuad Tabba, Joey Gouly, Suzuki K Poulose, Zenghui Yu, Shuah Khan,
James Clark, Alexandru Elisei, Mingwei Zhang, Ganapatrao Kulkarni,
kvmarm, linux-arm-kernel, kvm, linux-perf-users, linux-kselftest,
linux-doc, linux-kernel
Extend arm64/vpmu_counter_access to run all PMU counter access,
PMCR.N validation, and overflow interrupt tests against both the
emulated vPMU and the Partitioned PMU.
Introduce enum pmu_impl (EMULATED vs. PARTITIONED) and thread it
through the test setup helpers to configure
KVM_ARM_VCPU_PMU_V3_ENABLE_PARTITION and
KVM_ARM_VCPU_PMU_V3_SET_NR_COUNTERS when testing the Partitioned PMU.
Exercise PMCCFILTR_EL0 and live PMCCNTR_EL0 counting before validating
unimplemented counters so that Partitioned PMU vCPUs transition to
VCPU_PMU_ACCESS_GUEST_OWNED even when PMCR_EL0.N == 0. On hosts without
FEAT_HPMN0, allow n == 0 whenever max_counters >= 2 and verify that
counter 0 (hidden behind traps while physical MDCR_EL2.HPMN == 1) still
raises UNDEFINED on direct and indirect accesses and is RAZ/WI in the
bitmap and software-increment registers.
Skip out-of-bounds counter access exception tests for other counters
when the PMU is partitioned, as hardware does not guarantee an UNDEFINED
exception on accesses to counters >= MDCR_EL2.HPMN.
Signed-off-by: Colton Lewis <coltonlewis@google.com>
---
.../selftests/kvm/arm64/vpmu_counter_access.c | 217 +++++++++++++++---
1 file changed, 181 insertions(+), 36 deletions(-)
diff --git a/tools/testing/selftests/kvm/arm64/vpmu_counter_access.c b/tools/testing/selftests/kvm/arm64/vpmu_counter_access.c
index 22223395969e0..9028d08806540 100644
--- a/tools/testing/selftests/kvm/arm64/vpmu_counter_access.c
+++ b/tools/testing/selftests/kvm/arm64/vpmu_counter_access.c
@@ -25,12 +25,28 @@
/* The cycle counter bit position that's common among the PMU registers */
#define ARMV8_PMU_CYCLE_IDX 31
+enum pmu_impl {
+ EMULATED,
+ PARTITIONED
+};
+
+const char *pmu_impl_str[] = {
+ "Emulated",
+ "Partitioned"
+};
+
struct vpmu_vm {
struct kvm_vm *vm;
struct kvm_vcpu *vcpu;
};
+struct guest_context {
+ bool pmu_partitioned;
+ bool has_hpmn0;
+};
+
static struct vpmu_vm vpmu_vm;
+static struct guest_context guest_context;
struct pmreg_sets {
u64 set_reg_id;
@@ -331,19 +347,78 @@ static void test_access_invalid_pmc_regs(struct pmc_accessor *acc, int pmc_idx)
/*
* Reading/writing the event count/type registers should cause
* an UNDEFINED exception.
+ *
+ * On a Partitioned PMU hardware does not guarantee UNDEFINED for
+ * counters >= MDCR_EL2.HPMN. However, when PMCR_EL0.N == 0 on a host
+ * without FEAT_HPMN0, KVM programs HPMN = 1 and hides counter 0 behind
+ * traps, so accessing counter 0 must still raise UNDEFINED.
*/
- TEST_EXCEPTION(ESR_ELx_EC_UNKNOWN, acc->read_cntr(pmc_idx));
- TEST_EXCEPTION(ESR_ELx_EC_UNKNOWN, acc->write_cntr(pmc_idx, 0));
- TEST_EXCEPTION(ESR_ELx_EC_UNKNOWN, acc->read_typer(pmc_idx));
- TEST_EXCEPTION(ESR_ELx_EC_UNKNOWN, acc->write_typer(pmc_idx, 0));
+ if (!guest_context.pmu_partitioned ||
+ (!guest_context.has_hpmn0 && pmc_idx == 0)) {
+ TEST_EXCEPTION(ESR_ELx_EC_UNKNOWN, acc->read_cntr(pmc_idx));
+ TEST_EXCEPTION(ESR_ELx_EC_UNKNOWN, acc->write_cntr(pmc_idx, 0));
+ TEST_EXCEPTION(ESR_ELx_EC_UNKNOWN, acc->read_typer(pmc_idx));
+ TEST_EXCEPTION(ESR_ELx_EC_UNKNOWN, acc->write_typer(pmc_idx, 0));
+ }
/*
- * The bit corresponding to the (unimplemented) counter in
- * {PMCNTEN,PMINTEN,PMOVS}{SET,CLR} registers should be RAZ.
+ * Software increment on an unimplemented counter should be WI, and the
+ * bit corresponding to the counter in {PMCNTEN,PMINTEN,PMOVS}{SET,CLR}
+ * registers should be RAZ.
*/
+ write_sysreg(BIT(pmc_idx), pmswinc_el0);
+ isb();
test_bitmap_pmu_regs(pmc_idx, 1);
test_bitmap_pmu_regs(pmc_idx, 0);
}
+static void test_cycle_counter(u64 expected_pmcr_n)
+{
+ u64 filtr = ARMV8_PMU_EXCLUDE_EL0;
+ u64 pmcr, val;
+ int i;
+
+ if (get_current_el() == 2)
+ filtr |= ARMV8_PMU_INCLUDE_EL2;
+
+ /*
+ * Verify PMCCFILTR_EL0 read/write and live PMCCNTR_EL0 counting.
+ * On Partitioned PMU this also transitions the vCPU to
+ * VCPU_PMU_ACCESS_GUEST_OWNED even when expected_pmcr_n == 0 so
+ * subsequent unimplemented counter checks exercise hardware traps.
+ */
+ write_sysreg(filtr, pmccfiltr_el0);
+ isb();
+ val = read_sysreg(pmccfiltr_el0);
+ __GUEST_ASSERT(val == filtr,
+ "Unexpected PMCCFILTR_EL0: 0x%lx (expected 0x%lx)",
+ val, filtr);
+
+ pmcr = read_sysreg(pmcr_el0) & ~ARMV8_PMU_PMCR_E;
+ write_sysreg(pmcr, pmcr_el0);
+ write_sysreg(0, pmccntr_el0);
+ write_sysreg(BIT(ARMV8_PMU_CYCLE_IDX), pmcntenset_el0);
+ isb();
+
+ write_sysreg(pmcr | ARMV8_PMU_PMCR_E, pmcr_el0);
+ isb();
+
+ __GUEST_ASSERT(get_pmcr_n(read_sysreg(pmcr_el0)) == expected_pmcr_n,
+ "Expected PMCR.N: 0x%lx, PMCR.N: 0x%lx",
+ expected_pmcr_n, get_pmcr_n(read_sysreg(pmcr_el0)));
+
+ for (i = 0; i < 10000; i++)
+ asm volatile("nop");
+ isb();
+
+ val = read_sysreg(pmccntr_el0);
+ __GUEST_ASSERT(val > 0, "Cycle counter did not advance (0x%lx)", val);
+
+ write_sysreg(pmcr, pmcr_el0);
+ write_sysreg(BIT(ARMV8_PMU_CYCLE_IDX), pmcntenclr_el0);
+ write_sysreg(BIT(ARMV8_PMU_CYCLE_IDX), pmovsclr_el0);
+ isb();
+}
+
/*
* The guest is configured with PMUv3 with @expected_pmcr_n number of
* event counters.
@@ -386,6 +461,8 @@ static void guest_code(u64 expected_pmcr_n)
test_access_pmc_regs(&pmc_accessors[i], pmc);
}
+ test_cycle_counter(expected_pmcr_n);
+
/*
* Tests for reading/writing PMU registers for unimplemented counters.
* Use each combination of PMEV{CNTR,TYPER}<n>_EL0 accessor functions.
@@ -399,7 +476,7 @@ static void guest_code(u64 expected_pmcr_n)
}
/* Create a VM that has one vCPU with PMUv3 configured. */
-static void create_vpmu_vm(void *guest_code)
+static void create_vpmu_vm(void *guest_code, enum pmu_impl impl)
{
struct kvm_vcpu_init init;
u8 pmuver, ec;
@@ -409,6 +486,13 @@ static void create_vpmu_vm(void *guest_code)
.attr = KVM_ARM_VCPU_PMU_V3_IRQ,
.addr = (u64)&irq,
};
+ u32 partition = (impl == PARTITIONED);
+ struct kvm_device_attr part_attr = {
+ .group = KVM_ARM_VCPU_PMU_V3_CTRL,
+ .attr = KVM_ARM_VCPU_PMU_V3_ENABLE_PARTITION,
+ .addr = (uint64_t)&partition
+ };
+ int ret;
/* The test creates the vpmu_vm multiple times. Ensure a clean state */
memset(&vpmu_vm, 0, sizeof(vpmu_vm));
@@ -434,8 +518,19 @@ static void create_vpmu_vm(void *guest_code)
TEST_ASSERT(pmuver != ID_AA64DFR0_EL1_PMUVer_IMP_DEF &&
pmuver >= ID_AA64DFR0_EL1_PMUVer_IMP,
"Unexpected PMUVER (0x%x) on the vCPU with PMUv3", pmuver);
+ guest_context.has_hpmn0 = FIELD_GET(ID_AA64DFR0_EL1_HPMN0, dfr0) >=
+ ID_AA64DFR0_EL1_HPMN0_IMP;
vcpu_ioctl(vpmu_vm.vcpu, KVM_SET_DEVICE_ATTR, &irq_attr);
+
+ ret = __vcpu_has_device_attr(vpmu_vm.vcpu, KVM_ARM_VCPU_PMU_V3_CTRL,
+ KVM_ARM_VCPU_PMU_V3_ENABLE_PARTITION);
+ if (!ret) {
+ vcpu_ioctl(vpmu_vm.vcpu, KVM_SET_DEVICE_ATTR, &part_attr);
+ guest_context.pmu_partitioned = partition;
+ pr_debug("Set PMU partitioning: %d\n", partition);
+ }
+
}
static void destroy_vpmu_vm(void)
@@ -461,13 +556,15 @@ static void run_vcpu(struct kvm_vcpu *vcpu, u64 pmcr_n)
}
}
-static void test_create_vpmu_vm_with_nr_counters(unsigned int nr_counters, bool expect_fail)
+static void test_create_vpmu_vm_with_nr_counters(unsigned int nr_counters,
+ enum pmu_impl impl,
+ bool expect_fail)
{
struct kvm_vcpu *vcpu;
unsigned int prev;
int ret;
- create_vpmu_vm(guest_code);
+ create_vpmu_vm(guest_code, impl);
vcpu = vpmu_vm.vcpu;
prev = get_pmcr_n(vcpu_get_reg(vcpu, KVM_ARM64_SYS_REG(SYS_PMCR_EL0)));
@@ -475,21 +572,24 @@ static void test_create_vpmu_vm_with_nr_counters(unsigned int nr_counters, bool
ret = __vcpu_device_attr_set(vcpu, KVM_ARM_VCPU_PMU_V3_CTRL,
KVM_ARM_VCPU_PMU_V3_SET_NR_COUNTERS, &nr_counters);
- if (expect_fail)
+ if (expect_fail) {
TEST_ASSERT(ret && errno == EINVAL,
- "Setting more PMU counters (%u) than available (%u) unexpectedly succeeded",
+ "Setting invalid PMU counter count (%u, default %u) unexpectedly succeeded",
nr_counters, prev);
- else
- TEST_ASSERT(!ret, KVM_IOCTL_ERROR(KVM_SET_DEVICE_ATTR, ret));
+ return;
+ }
+
+ TEST_ASSERT(!ret, KVM_IOCTL_ERROR(KVM_SET_DEVICE_ATTR, ret));
vcpu_device_attr_set(vcpu, KVM_ARM_VCPU_PMU_V3_CTRL, KVM_ARM_VCPU_PMU_V3_INIT, NULL);
+ sync_global_to_guest(vpmu_vm.vm, guest_context);
}
/*
* Create a guest with one vCPU, set the PMCR_EL0.N for the vCPU to @pmcr_n,
* and run the test.
*/
-static void run_access_test(u64 pmcr_n)
+static void run_access_test(u64 pmcr_n, enum pmu_impl impl)
{
u64 sp;
struct kvm_vcpu *vcpu;
@@ -497,7 +597,7 @@ static void run_access_test(u64 pmcr_n)
pr_debug("Test with pmcr_n %lu\n", pmcr_n);
- test_create_vpmu_vm_with_nr_counters(pmcr_n, false);
+ test_create_vpmu_vm_with_nr_counters(pmcr_n, impl, false);
vcpu = vpmu_vm.vcpu;
/* Save the initial sp to restore them later to run the guest again */
@@ -531,14 +631,14 @@ static struct pmreg_sets validity_check_reg_sets[] = {
* Create a VM, and check if KVM handles the userspace accesses of
* the PMU register sets in @validity_check_reg_sets[] correctly.
*/
-static void run_pmregs_validity_test(u64 pmcr_n)
+static void run_pmregs_validity_test(u64 pmcr_n, enum pmu_impl impl)
{
int i;
struct kvm_vcpu *vcpu;
u64 set_reg_id, clr_reg_id, reg_val;
u64 valid_counters_mask, max_counters_mask;
- test_create_vpmu_vm_with_nr_counters(pmcr_n, false);
+ test_create_vpmu_vm_with_nr_counters(pmcr_n, impl, false);
vcpu = vpmu_vm.vcpu;
valid_counters_mask = get_counters_mask(pmcr_n);
@@ -585,14 +685,13 @@ static void run_pmregs_validity_test(u64 pmcr_n)
/*
* Create a guest with one vCPU, and attempt to set the PMCR_EL0.N for
- * the vCPU to @pmcr_n, which is larger than the host value.
- * The attempt should fail as @pmcr_n is too big to set for the vCPU.
+ * the vCPU to an invalid @pmcr_n value.
*/
-static void run_error_test(u64 pmcr_n)
+static void run_error_test(u64 pmcr_n, enum pmu_impl impl)
{
- pr_debug("Error test with pmcr_n %lu (larger than the host)\n", pmcr_n);
+ pr_debug("Error test with pmcr_n %lu\n", pmcr_n);
- test_create_vpmu_vm_with_nr_counters(pmcr_n, true);
+ test_create_vpmu_vm_with_nr_counters(pmcr_n, impl, true);
destroy_vpmu_vm();
}
@@ -600,21 +699,26 @@ static void run_error_test(u64 pmcr_n)
* Return the default number of implemented PMU event counters excluding
* the cycle counter (i.e. PMCR_EL0.N value) for the guest.
*/
-static u64 get_pmcr_n_limit(void)
+static u64 get_pmcr_n_limit(enum pmu_impl impl)
{
- u64 pmcr;
+ u64 pmcr, n;
- create_vpmu_vm(guest_code);
+ create_vpmu_vm(guest_code, impl);
pmcr = vcpu_get_reg(vpmu_vm.vcpu, KVM_ARM64_SYS_REG(SYS_PMCR_EL0));
destroy_vpmu_vm();
- return get_pmcr_n(pmcr);
+
+ n = get_pmcr_n(pmcr);
+ if (impl == PARTITIONED && n > 0)
+ return n - 1;
+
+ return n;
}
static bool kvm_supports_nr_counters_attr(void)
{
bool supported;
- create_vpmu_vm(NULL);
+ create_vpmu_vm(NULL, EMULATED);
supported = !__vcpu_has_device_attr(vpmu_vm.vcpu, KVM_ARM_VCPU_PMU_V3_CTRL,
KVM_ARM_VCPU_PMU_V3_SET_NR_COUNTERS);
destroy_vpmu_vm();
@@ -622,22 +726,63 @@ static bool kvm_supports_nr_counters_attr(void)
return supported;
}
-int main(void)
+static bool kvm_supports_partition_attr(void)
{
- u64 i, pmcr_n;
+ bool supported;
+ create_vpmu_vm(NULL, EMULATED);
+ supported = !__vcpu_has_device_attr(vpmu_vm.vcpu, KVM_ARM_VCPU_PMU_V3_CTRL,
+ KVM_ARM_VCPU_PMU_V3_ENABLE_PARTITION);
+ destroy_vpmu_vm();
+
+ return supported;
+}
+
+static bool kvm_supports_hpmn0(void)
+{
+ u64 dfr0;
+
+ create_vpmu_vm(NULL, EMULATED);
+ dfr0 = vcpu_get_reg(vpmu_vm.vcpu, KVM_ARM64_SYS_REG(SYS_ID_AA64DFR0_EL1));
+ destroy_vpmu_vm();
+
+ return FIELD_GET(ID_AA64DFR0_EL1_HPMN0, dfr0) >= ID_AA64DFR0_EL1_HPMN0_IMP;
+}
+
+void test_pmu(enum pmu_impl impl)
+{
+ u64 i, min_n = 0, pmcr_n;
+
+ pr_info("Testing PMU: Implementation = %s\n", pmu_impl_str[impl]);
+
+ pmcr_n = get_pmcr_n_limit(impl);
+ pr_debug("PMCR_EL0.N: Limit = %lu\n", pmcr_n);
+
+ if (impl == PARTITIONED && !kvm_supports_hpmn0() &&
+ get_pmcr_n_limit(EMULATED) < 2) {
+ run_error_test(0, impl);
+ min_n = 1;
+ }
+
+ for (i = min_n; i <= pmcr_n; i++) {
+ run_access_test(i, impl);
+ run_pmregs_validity_test(i, impl);
+ }
+
+ for (i = pmcr_n + 1; i < ARMV8_PMU_MAX_COUNTERS; i++)
+ run_error_test(i, impl);
+}
+
+int main(void)
+{
TEST_REQUIRE(kvm_has_cap(KVM_CAP_ARM_PMU_V3));
TEST_REQUIRE(kvm_supports_vgic_v3());
TEST_REQUIRE(kvm_supports_nr_counters_attr());
- pmcr_n = get_pmcr_n_limit();
- for (i = 0; i <= pmcr_n; i++) {
- run_access_test(i);
- run_pmregs_validity_test(i);
- }
+ test_pmu(EMULATED);
- for (i = pmcr_n + 1; i < ARMV8_PMU_MAX_COUNTERS; i++)
- run_error_test(i);
+ if (kvm_supports_partition_attr() && get_pmcr_n_limit(EMULATED) > 0)
+ test_pmu(PARTITIONED);
return 0;
}
--
2.56.1.708.g77570a6a1d-goog
^ permalink raw reply related [flat|nested] 67+ messages in thread
* Re: [PATCH v10 15/33] KVM: arm64: Construct mdcr_el2 in a local variable in kvm_arm_setup_mdcr_el2()
2026-11-19 18:51 ` [PATCH v10 15/33] KVM: arm64: Construct mdcr_el2 in a local variable in kvm_arm_setup_mdcr_el2() Colton Lewis
@ 2026-11-19 18:56 ` sashiko-bot
0 siblings, 0 replies; 67+ messages in thread
From: sashiko-bot @ 2026-11-19 18:56 UTC (permalink / raw)
To: Colton Lewis; +Cc: linux-perf-users
> Construct the new MDCR_EL2 value in a local variable before writing it
> once to vcpu->arch.mdcr_el2. This avoids exposing intermediate states in
> vcpu->arch.mdcr_el2 to interrupt or NMI handlers that may inspect it,
> and simplifies upcoming Partitioned PMU modifications to MDCR_EL2.
>
> Signed-off-by: Colton Lewis <coltonlewis@google.com>
Sashiko has reviewed this patch and found no issues.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261119185215.632334-1-coltonlewis@google.com?part=15
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v10 06/33] perf: arm_pmuv3: Generalize counter bitmasks
2026-11-19 18:51 ` [PATCH v10 06/33] perf: arm_pmuv3: Generalize counter bitmasks Colton Lewis
@ 2026-11-19 18:56 ` sashiko-bot
0 siblings, 0 replies; 67+ messages in thread
From: sashiko-bot @ 2026-11-19 18:56 UTC (permalink / raw)
To: Colton Lewis; +Cc: linux-perf-users
> The OVSR bitmasks are valid for enable and interrupt registers as well as
> overflow registers. Generalize the names.
>
> Acked-by: Mark Rutland <mark.rutland@arm.com>
> Tested-by: James Clark <james.clark@linaro.org>
> Signed-off-by: Colton Lewis <coltonlewis@google.com>
Sashiko has reviewed this patch and found no issues.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261119185215.632334-1-coltonlewis@google.com?part=6
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v10 10/33] perf: arm_pmuv3: Expose number of general-purpose counters in sysfs
2026-11-19 18:51 ` [PATCH v10 10/33] perf: arm_pmuv3: Expose number of general-purpose counters in sysfs Colton Lewis
@ 2026-11-19 18:57 ` sashiko-bot
0 siblings, 0 replies; 67+ messages in thread
From: sashiko-bot @ 2026-11-19 18:57 UTC (permalink / raw)
To: Colton Lewis; +Cc: linux-perf-users
> Expose the number of general-purpose event counters implemented by each
> ARMv8 PMUv3 instance at
> /sys/bus/event_source/devices/<pmu>/caps/num_counters.
>
> This allows userspace (such as VMMs configuring
> KVM_ARM_VCPU_PMU_V3_SET_NR_COUNTERS for a Partitioned PMU on
> heterogeneous systems) to query the maximum number of general-purpose
> counters for a specific PMU without reading PMCR_EL0.N directly from
> EL0.
>
> Suggested-by: James Clark <james.clark@linaro.org>
> Signed-off-by: Colton Lewis <coltonlewis@google.com>
Sashiko has reviewed this patch and found no issues.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261119185215.632334-1-coltonlewis@google.com?part=10
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v10 03/33] KVM: arm64: Reorganize PMU functions
2026-11-19 18:51 ` [PATCH v10 03/33] KVM: arm64: Reorganize PMU functions Colton Lewis
@ 2026-11-19 18:57 ` sashiko-bot
0 siblings, 0 replies; 67+ messages in thread
From: sashiko-bot @ 2026-11-19 18:57 UTC (permalink / raw)
To: Colton Lewis; +Cc: linux-perf-users
> A lot of functions in pmu-emul.c aren't specific to the emulated PMU
> implementation. Move them to the more appropriate pmu.c file where
> shared PMU functions should live.
>
> Tested-by: James Clark <james.clark@linaro.org>
> Signed-off-by: Colton Lewis <coltonlewis@google.com>
Sashiko has reviewed this patch and found no issues.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261119185215.632334-1-coltonlewis@google.com?part=3
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v10 09/33] perf: arm_pmuv3: Allocate counter indices from high to low
2026-11-19 18:51 ` [PATCH v10 09/33] perf: arm_pmuv3: Allocate counter indices from high to low Colton Lewis
@ 2026-11-19 18:57 ` sashiko-bot
0 siblings, 0 replies; 67+ messages in thread
From: sashiko-bot @ 2026-11-19 18:57 UTC (permalink / raw)
To: Colton Lewis; +Cc: linux-perf-users
> MDCR_EL2.HPMN assigns low-index general-purpose counters (0..HPMN-1) to
> the guest and high-index counters (HPMN..N-1) to the host. Allocate host
> event counters from high index to low index to minimize collisions with
> guest counter partitions and avoid rescheduling host perf events on vCPU
> load when enough upper counters are free.
>
> When allocating a 64-bit chained pair (even counter idx - 1 paired with
> odd counter idx), verify that both idx and idx - 1 are present in
> cpuc->cntr_mask so an odd HPMN boundary cannot allocate a guest-owned
> even counter as the lower half of a host chained event.
>
> Tested-by: James Clark <james.clark@linaro.org>
> Signed-off-by: Colton Lewis <coltonlewis@google.com>
Sashiko has reviewed this patch and found no issues.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261119185215.632334-1-coltonlewis@google.com?part=9
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v10 11/33] KVM: arm64: Add initial scaffolding for Partitioned PMU
2026-11-19 18:51 ` [PATCH v10 11/33] KVM: arm64: Add initial scaffolding for Partitioned PMU Colton Lewis
@ 2026-11-19 18:58 ` sashiko-bot
0 siblings, 0 replies; 67+ messages in thread
From: sashiko-bot @ 2026-11-19 18:58 UTC (permalink / raw)
To: Colton Lewis; +Cc: linux-perf-users
> Add the foundational helpers for Partitioned PMU support in
> arch/arm64/kvm/pmu-direct.c:
> - has_kvm_pmu_partition_support() to check VHE and PMUv3 availability.
> - kvm_pmu_partition_enable() and kvm_pmu_is_partitioned() to track per-VM
> partitioning state via KVM_ARCH_FLAG_PARTITION_PMU_ENABLED.
> - kvm_pmu_guest_counter_mask() to compute the bitmask of guest-reserved
> (0..HPMN-1 and PMCCNTR_EL0) hardware counters.
>
> Tested-by: James Clark <james.clark@linaro.org>
> Signed-off-by: Colton Lewis <coltonlewis@google.com>
Sashiko has reviewed this patch and found no issues.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261119185215.632334-1-coltonlewis@google.com?part=11
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v10 13/33] KVM: arm64: Make __vcpu_rmw_sys_reg() atomic
2026-11-19 18:51 ` [PATCH v10 13/33] KVM: arm64: Make __vcpu_rmw_sys_reg() atomic Colton Lewis
@ 2026-11-19 18:58 ` sashiko-bot
0 siblings, 0 replies; 67+ messages in thread
From: sashiko-bot @ 2026-11-19 18:58 UTC (permalink / raw)
To: Colton Lewis; +Cc: linux-perf-users
> Use try_cmpxchg_local() in __vcpu_rmw_sys_reg() so read-modify-write
> updates to vCPU system registers are atomic with respect to local
> interrupts and GICv3 pseudo-NMIs. This prevents lost updates when a PMU
> overflow handler (which may run as a pseudo-NMI that is not masked by
> local_irq_save()) updates PMOVSSET_EL0 concurrently with vCPU register
> accesses.
>
> Signed-off-by: Colton Lewis <coltonlewis@google.com>
Sashiko has reviewed this patch and found no issues.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261119185215.632334-1-coltonlewis@google.com?part=13
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v10 05/33] KVM: arm64: Ignore HPMN for userspace PMU register access
2026-11-19 18:51 ` [PATCH v10 05/33] KVM: arm64: Ignore HPMN for userspace PMU register access Colton Lewis
@ 2026-11-19 18:58 ` sashiko-bot
0 siblings, 0 replies; 67+ messages in thread
From: sashiko-bot @ 2026-11-19 18:58 UTC (permalink / raw)
To: Colton Lewis; +Cc: linux-perf-users
> Userspace save and restore (KVM_GET_ONE_REG / KVM_SET_ONE_REG) must
> observe and preserve all VM-implemented PMU counters
> ([0..nr_pmu_counters-1] plus PMCCNTR_EL0), independent of the vCPU's
> current exception level or MDCR_EL2.HPMN.
>
> Currently, two UAPI callbacks still depend on guest execution context
> when a nested vCPU (KVM_ARM_VCPU_HAS_EL2) is paused below EL2
> (!vcpu_is_el2(vcpu)) with MDCR_EL2.HPMN < nr_pmu_counters:
>
> - set_pmreg() and get_pmreg() (backing PMINTEN{SET,CLR}_EL1,
> PMCNTEN{SET,CLR}_EL0, and PMOVS{SET,CLR}_EL0) mask register values
> with kvm_pmu_accessible_counter_mask(), which strips L1 hypervisor
> counter bits [HPMN..N-1]. Saving a vCPU paused in L2 hides those bits,
> and restoring them while paused in L2 permanently clears them.
> - get_pmcr() calls kvm_vcpu_read_pmcr(), which substitutes
> MDCR_EL2.HPMN for PMCR_EL0.N below EL2, corrupting the saved
> PMCR_EL0.N value.
>
> Switch set_pmreg() and get_pmreg() back to
> kvm_pmu_implemented_counter_mask(), and have get_pmcr() populate
> PMCR_EL0.N directly from vcpu->kvm->arch.nr_pmu_counters using
> u64_replace_bits(). Keep kvm_vcpu_read_pmcr() and
> kvm_pmu_accessible_counter_mask() for guest trap paths.
>
> Fixes: 9a1c58cfefb0 ("KVM: arm64: nv: Adjust range of accessible PMCs according to HPMN")
> Fixes: 600f6fa5c90c ("KVM: arm64: Let kvm_vcpu_read_pmcr() return an EL-dependent value for PMCR_EL0.N")
> Signed-off-by: Colton Lewis <coltonlewis@google.com>
Sashiko has reviewed this patch and found no issues.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261119185215.632334-1-coltonlewis@google.com?part=5
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v10 02/33] KVM: arm64: Reorganize PMU includes
2026-11-19 18:51 ` [PATCH v10 02/33] KVM: arm64: Reorganize PMU includes Colton Lewis
@ 2026-11-19 19:00 ` sashiko-bot
0 siblings, 0 replies; 67+ messages in thread
From: sashiko-bot @ 2026-11-19 19:00 UTC (permalink / raw)
To: Colton Lewis; +Cc: linux-perf-users
> From: Marc Zyngier <maz@kernel.org>
>
> Including *all* of asm/kvm_host.h in asm/arm_pmuv3.h is a bad idea
> because that is much more than arm_pmuv3.h logically needs and creates
> a circular dependency that makes it easy to introduce compiler errors
> when editing this code.
>
> asm/kvm_host.h includes kvm/arm_pmu.h includes perf/arm_pmuv3.h
> includes asm/arm_pmuv3.h includes asm/kvm_host.h
>
> Reorganize the PMU includes to be more sane. In particular:
>
> - Remove the circular dependency by removing the kvm_host.h include
> from asm/arm_pmuv3.h since 99% of it isn't needed.
>
> - Move the remaining tiny bit of KVM/PMU interface from kvm_host.h
> into arm_pmu.h
>
> - Conditionally on ARM64, include the more targeted arm_pmu.h directly
> in the arm_pmuv3.c driver.
>
> Signed-off-by: Marc Zyngier <maz@kernel.org>
> Tested-by: James Clark <james.clark@linaro.org>
> Signed-off-by: Colton Lewis <coltonlewis@google.com>
Sashiko has reviewed this patch and found no issues.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261119185215.632334-1-coltonlewis@google.com?part=2
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v10 20/33] perf/core: Prioritize kernel-owned pinned events when scheduling groups
2026-11-19 18:52 ` [PATCH v10 20/33] perf/core: Prioritize kernel-owned pinned events when scheduling groups Colton Lewis
@ 2026-11-19 19:01 ` sashiko-bot
0 siblings, 0 replies; 67+ messages in thread
From: sashiko-bot @ 2026-11-19 19:01 UTC (permalink / raw)
To: Colton Lewis; +Cc: linux-perf-users
> When scheduling event groups in visit_groups_merge(), pinned and
> flexible groups are ordered by group_index (creation order). Normally,
> kernel-owned pinned events such as the hardlockup NMI watchdog are
> created at boot with group_index 0 and are scheduled ahead of userspace
> events.
>
> However, if the NMI watchdog is disabled and re-enabled at runtime
> after a userspace pinned per-CPU event is already active, the
> watchdog's group_index is larger than the userspace event's
> group_index. When hardware counters are constrained (for example, when
> PMU partitioning reserves counters for a guest and leaves only one
> general-purpose counter for the host), the userspace pinned event
> claims the remaining counter first and merge_sched_in() transitions the
> kernel watchdog event into PERF_EVENT_STATE_ERROR.
>
> Order pinned kernel-owned events (is_pinned_kernel_event()) ahead of
> userspace pinned events in perf_event_groups_cmp() and
> perf_less_group_idx() while leaving flexible groups ordered by
> group_index for normal round-robin rotation.
>
> Suggested-by: James Clark <james.clark@linaro.org>
> Signed-off-by: Colton Lewis <coltonlewis@google.com>
Sashiko has reviewed this patch and found no issues.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261119185215.632334-1-coltonlewis@google.com?part=20
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v10 08/33] perf: arm_pmuv3: Check cntr_mask before using pmccntr
2026-11-19 18:51 ` [PATCH v10 08/33] perf: arm_pmuv3: Check cntr_mask before using pmccntr Colton Lewis
@ 2026-11-19 19:01 ` sashiko-bot
0 siblings, 0 replies; 67+ messages in thread
From: sashiko-bot @ 2026-11-19 19:01 UTC (permalink / raw)
To: Colton Lewis; +Cc: linux-perf-users
> When the PMU is partitioned, PMCCNTR_EL0 is reserved for the guest and
> removed from cpuc->cntr_mask. Move the PMCCNTR_EL0 availability checks
> (cpuc->cntr_mask and cpuc->used_mask) into armv8pmu_can_use_pmccntr()
> and un-nest the 64-bit user-access fallback check in
> armv8pmu_get_event_idx() so host CPU_CYCLES events fall back to
> general-purpose event counters when PMCCNTR_EL0 is unavailable.
>
> Tested-by: James Clark <james.clark@linaro.org>
> Signed-off-by: Colton Lewis <coltonlewis@google.com>
Sashiko has reviewed this patch and found no issues.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261119185215.632334-1-coltonlewis@google.com?part=8
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v10 19/33] perf: Add perf_pmu_resched_update()
2026-11-19 18:52 ` [PATCH v10 19/33] perf: Add perf_pmu_resched_update() Colton Lewis
@ 2026-11-19 19:02 ` sashiko-bot
0 siblings, 0 replies; 67+ messages in thread
From: sashiko-bot @ 2026-11-19 19:02 UTC (permalink / raw)
To: Colton Lewis; +Cc: linux-perf-users
> To modify PMU guest counter reservations dynamically, the per-CPU
> available counter mask must be updated while host perf events are
> temporarily scheduled out.
>
> Introduce perf_pmu_resched_update() to update PMU state while holding
> cpuctx->ctx.lock in between scheduling perf events out and scheduling
> them back in. It accepts a callback invoked between ctx_sched_out() and
> perf_event_sched_in(), accomplishing atomic counter mask updates with
> minimal perf API expansion.
>
> Refactor ctx_resched() into __ctx_resched() to invoke the callback at
> the appropriate point.
>
> Tested-by: James Clark <james.clark@linaro.org>
> Signed-off-by: Colton Lewis <coltonlewis@google.com>
Sashiko has reviewed this patch and found no issues.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261119185215.632334-1-coltonlewis@google.com?part=19
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v10 07/33] perf: arm_pmuv3: Move counter allocation mask to per-CPU struct pmu_hw_events
2026-11-19 18:51 ` [PATCH v10 07/33] perf: arm_pmuv3: Move counter allocation mask to per-CPU struct pmu_hw_events Colton Lewis
@ 2026-11-19 19:04 ` sashiko-bot
0 siblings, 0 replies; 67+ messages in thread
From: sashiko-bot @ 2026-11-19 19:04 UTC (permalink / raw)
To: Colton Lewis; +Cc: linux-perf-users
> In preparation for dynamic per-CPU PMU counter reservations when running
> KVM guests with a partitioned PMU, move the counter allocation mask from
> the global struct arm_pmu to per-CPU struct pmu_hw_events.
>
> Initialize cpuc->cntr_mask from the static cpu_pmu->cntr_mask during PMU
> probe and CPU hotplug startup, and update event counter allocation
> helpers (armv8pmu_get_single_idx(), armv8pmu_get_chain_idx(), and
> armv8pmu_get_event_idx()) to query the per-CPU mask.
>
> Tested-by: James Clark <james.clark@linaro.org>
> Signed-off-by: Colton Lewis <coltonlewis@google.com>
Sashiko has reviewed this patch and found no issues.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261119185215.632334-1-coltonlewis@google.com?part=7
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v10 21/33] KVM: arm64: Allow kvm_vcpu_pmu_resync_el0() to resync filters in process context
2026-11-19 18:52 ` [PATCH v10 21/33] KVM: arm64: Allow kvm_vcpu_pmu_resync_el0() to resync filters in process context Colton Lewis
@ 2026-11-19 19:07 ` sashiko-bot
0 siblings, 0 replies; 67+ messages in thread
From: sashiko-bot @ 2026-11-19 19:07 UTC (permalink / raw)
To: Colton Lewis; +Cc: linux-perf-users
> Previously, kvm_vcpu_pmu_resync_el0() only scheduled
> KVM_REQ_RESYNC_PMU_EL0 when called from interrupt context (such as an
> IPI reprogramming host perf events). When perf_pmu_resched_update() is
> invoked synchronously from process context during vCPU load/put to
> reschedule host events around guest-reserved counters,
> armv8pmu_enable_event() calls kvm_set_pmu_events() and
> kvm_vcpu_pmu_resync_el0() in process context.
>
> Update kvm_vcpu_pmu_resync_el0() to call kvm_vcpu_pmu_restore_guest()
> directly when not in interrupt context so that newly rescheduled host
> perf events have their EL0 exclusion filters applied immediately.
>
> Tested-by: James Clark <james.clark@linaro.org>
> Signed-off-by: Colton Lewis <coltonlewis@google.com>
Sashiko has reviewed this patch and found no issues.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261119185215.632334-1-coltonlewis@google.com?part=21
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v10 32/33] KVM: selftests: Add find_bit to KVM library
2026-11-19 18:52 ` [PATCH v10 32/33] KVM: selftests: Add find_bit to KVM library Colton Lewis
@ 2026-11-19 19:07 ` sashiko-bot
0 siblings, 0 replies; 67+ messages in thread
From: sashiko-bot @ 2026-11-19 19:07 UTC (permalink / raw)
To: Colton Lewis; +Cc: linux-perf-users
> Add lib/find_bit.c to the KVM selftests build using the same wrapper
> pattern as lib/rbtree.c so selftests that depend on find_bit helpers
> link cleanly when built standalone.
>
> Tested-by: James Clark <james.clark@linaro.org>
> Signed-off-by: Colton Lewis <coltonlewis@google.com>
Sashiko has reviewed this patch and found no issues.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261119185215.632334-1-coltonlewis@google.com?part=32
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v10 01/33] arm64: cpufeature: Add cpucap for HPMN0
2026-11-19 18:51 ` [PATCH v10 01/33] arm64: cpufeature: Add cpucap for HPMN0 Colton Lewis
@ 2026-11-19 19:08 ` sashiko-bot
0 siblings, 0 replies; 67+ messages in thread
From: sashiko-bot @ 2026-11-19 19:08 UTC (permalink / raw)
To: Colton Lewis; +Cc: linux-perf-users
> Add a capability for FEAT_HPMN0, whether MDCR_EL2.HPMN can specify 0
> counters reserved for the guest.
>
> This required changing HPMN0 to an UnsignedEnum in tools/sysreg
> because otherwise not all the appropriate macros are generated to add
> it to arm64_cpu_capabilities_arm64_features.
>
> Acked-by: Mark Rutland <mark.rutland@arm.com>
> Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>
> Tested-by: James Clark <james.clark@linaro.org>
> Signed-off-by: Colton Lewis <coltonlewis@google.com>
Sashiko has reviewed this patch and found no issues.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261119185215.632334-1-coltonlewis@google.com?part=1
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v10 04/33] KVM: arm64: Use nr_pmu_counters in kvm_pmu_implemented_counter_mask()
2026-11-19 18:51 ` [PATCH v10 04/33] KVM: arm64: Use nr_pmu_counters in kvm_pmu_implemented_counter_mask() Colton Lewis
@ 2026-11-19 19:08 ` sashiko-bot
0 siblings, 0 replies; 67+ messages in thread
From: sashiko-bot @ 2026-11-19 19:08 UTC (permalink / raw)
To: Colton Lewis; +Cc: linux-perf-users
> kvm_pmu_implemented_counter_mask() returns the bitmask of all counters
> implemented by the VM (0..N-1 and PMCCNTR_EL0). Previously it extracted
> N from kvm_vcpu_read_pmcr(vcpu), which commit 600f6fa5c90c ("KVM: arm64:
> Let kvm_vcpu_read_pmcr() return an EL-dependent value for PMCR_EL0.N")
> changed to substitute MDCR_EL2.HPMN for PMCR_EL0.N whenever
> vcpu_has_nv(vcpu) && !vcpu_is_el2(vcpu).
>
> As a result, when kvm_vcpu_reload_pmu() services KVM_REQ_RELOAD_PMU
> while the vCPU is at vEL1 or vEL0 (for example, when entering a nested
> guest after an L1 MDCR_EL2 update), kvm_pmu_implemented_counter_mask()
> omits HPMN..N-1 and kvm_vcpu_reload_pmu() wipes L1's hypervisor counter
> bits in PMOVSSET_EL0, PMINTENSET_EL1, and PMCNTENSET_EL0.
>
> Fix this in three parts:
> - Read vcpu->kvm->arch.nr_pmu_counters directly in
> kvm_pmu_implemented_counter_mask() so the VM-wide implemented mask is
> independent of the vCPU's current exception level.
> - Use kvm_pmu_accessible_counter_mask(vcpu) & ~BIT(ARMV8_PMU_CYCLE_IDX)
> for PMCR_EL0.P event counter resets instead of open-coding the
> subtraction of kvm_pmu_hyp_counter_mask(vcpu) below EL2.
> - Mask trapped reads in access_pmcnten(), access_pminten(), and
> access_pmovs() with kvm_pmu_accessible_counter_mask(vcpu) (matching
> existing write masking) so retained hypervisor counter bits in
> HPMN..N-1 read as zero at EL1 and EL0.
>
> Fixes: 600f6fa5c90c ("KVM: arm64: Let kvm_vcpu_read_pmcr() return an EL-dependent value for PMCR_EL0.N")
> Fixes: 9a1c58cfefb0 ("KVM: arm64: nv: Adjust range of accessible PMCs according to HPMN")
> Signed-off-by: Colton Lewis <coltonlewis@google.com>
Sashiko has reviewed this patch and found no issues.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261119185215.632334-1-coltonlewis@google.com?part=4
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v10 12/33] KVM: arm64: Set up FGT for Partitioned PMU
2026-11-19 18:51 ` [PATCH v10 12/33] KVM: arm64: Set up FGT " Colton Lewis
@ 2026-11-19 19:13 ` sashiko-bot
0 siblings, 0 replies; 67+ messages in thread
From: sashiko-bot @ 2026-11-19 19:13 UTC (permalink / raw)
To: Colton Lewis; +Cc: linux-perf-users
> In order to gain the best performance benefit from partitioning the
> PMU, utilize fine grain traps (FEAT_FGT and FEAT_FGT2) to avoid
> trapping common PMU register accesses by the guest to remove that
> overhead.
>
> Untrapped:
> - PMCR_EL0
> - PMUSERENR_EL0 (outside nested hypervisor context)
> - PMSELR_EL0
> - PMCCNTR_EL0
> - PMCNTEN_EL0
> - PMINTEN_EL1
> - PMEVCNTRn_EL0
>
> These are safe to untrap because writing MDCR_EL2.HPMN as this series
> [ ... ]
> disables EL0 PMU access, and trap PMUSERENR_EL0 reads via HDFGRTR_EL2
> when guest MDCR_EL2.TPM is set so MRS PMUSERENR_EL0 from VHE host EL0 is
> forwarded to L1.
>
> Signed-off-by: Colton Lewis <coltonlewis@google.com>
Sashiko has reviewed this patch and found no issues.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261119185215.632334-1-coltonlewis@google.com?part=12
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v10 28/33] KVM: arm64: Emulate trapped PMSWINC_EL0 writes for Partitioned PMU
2026-11-19 18:52 ` [PATCH v10 28/33] KVM: arm64: Emulate trapped PMSWINC_EL0 writes for Partitioned PMU Colton Lewis
@ 2026-11-19 19:13 ` sashiko-bot
0 siblings, 0 replies; 67+ messages in thread
From: sashiko-bot @ 2026-11-19 19:13 UTC (permalink / raw)
To: Colton Lewis; +Cc: linux-perf-users
> Even with a Partitioned PMU, guest writes to PMSWINC_EL0 trap to EL2 in
> four cases:
> 1. Hosts without FEAT_FGT, where MDCR_EL2.TPM remains set.
> 2. Before lazy PMU activation (VCPU_PMU_ACCESS_FREE, MDCR_EL2.TPM == 1).
> 3. Nested L2 execution when L1 sub-partitions counters (vHPMN < N_L1),
> where MDCR_EL2.TPM is forced to protect L1's hypervisor counters.
> 4. Nested L1 execution when L1 sub-partitions counters (vHPMN < N_L1),
> where HDFGWTR_EL2.PMSWINC_EL0 must trap software increments so an
> even counter at vHPMN - 1 cannot chain across the sub-partition
> boundary into L1 counter vHPMN in hardware, and so ovf32_mask
> counter biasing is preserved.
>
> Trapped PMSWINC_EL0 writes cannot simply execute MSR PMSWINC_EL0 at
> physical EL2 because L0 runs with physical MDCR_EL2.HPMD == 1 and clears
> PMEVTYPER<n>_EL0.NSH on guest counters, causing hardware to ignore the
> software increment (and when not loaded on the CPU, guest counters live
> only in sys_regs[]).
>
> Update kvm_pmu_counter_increment() and kvm_pmu_counter_can_chain() in
> arch/arm64/kvm/pmu-emul.c to check per-counter enablement
> (kvm_pmu_counter_is_enabled(), covering both PMCR_EL0.E and
> MDCR_EL2.HPME), current guest EL filtering for SW_INCR, and
> nr_pmu_counters / sub-partition chaining boundaries. Reuse
> kvm_pmu_software_increment() from kvm_pmu_direct_swinc() by flushing
> live hardware state if loaded, applying the software increment to
> sys_regs[], and reloading hardware.
>
> Signed-off-by: Colton Lewis <coltonlewis@google.com>
Sashiko has reviewed this patch and found no issues.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261119185215.632334-1-coltonlewis@google.com?part=28
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v10 31/33] KVM: arm64: Add vCPU device attr to partition the PMU
2026-11-19 18:52 ` [PATCH v10 31/33] KVM: arm64: Add vCPU device attr to partition the PMU Colton Lewis
@ 2026-11-19 19:14 ` sashiko-bot
0 siblings, 0 replies; 67+ messages in thread
From: sashiko-bot @ 2026-11-19 19:14 UTC (permalink / raw)
To: Colton Lewis; +Cc: linux-perf-users
> Add the KVM_ARM_VCPU_PMU_V3_ENABLE_PARTITION vCPU device attribute to
> enable a Partitioned PMU on hosts with VHE and PMUv3 support.
>
> When partitioning is enabled (KVM_ARCH_FLAG_PARTITION_PMU_ENABLED),
> userspace must configure guest general-purpose counters via
> KVM_ARM_VCPU_PMU_V3_SET_NR_COUNTERS before KVM_ARM_VCPU_PMU_V3_INIT:
> - Require kvm_arm_pmu_part_hpmn(n) < max_counters so at least one
> general-purpose counter remains available for host profiling.
> - Allow n == 0 (cycle counter only) on hosts without FEAT_HPMN0 when
> max_counters >= 2, reserving physical counter 0 (MDCR_EL2.HPMN = 1)
> and hiding it from the guest via traps.
>
> Because PMU attributes are invoked per-vCPU while arm_pmu,
> nr_pmu_counters, and partitioning state are VM-wide, VMMs such as QEMU
> iterate over vCPUs (CPU_FOREACH) and call KVM_ARM_VCPU_PMU_V3_INIT on
> vCPU 0 before configuring vCPU 1..N. Allow SET_PMU, SET_NR_COUNTERS, and
> ENABLE_PARTITION to succeed when a vCPU PMU is already initialized as
> long as the requested value matches the VM's existing configuration, and
> return -EBUSY on conflicting changes.
>
> Signed-off-by: Colton Lewis <coltonlewis@google.com>
Sashiko has reviewed this patch and found no issues.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261119185215.632334-1-coltonlewis@google.com?part=31
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v10 33/33] KVM: arm64: selftests: Add test case for Partitioned PMU
2026-11-19 18:52 ` [PATCH v10 33/33] KVM: arm64: selftests: Add test case for Partitioned PMU Colton Lewis
@ 2026-11-19 19:14 ` sashiko-bot
0 siblings, 0 replies; 67+ messages in thread
From: sashiko-bot @ 2026-11-19 19:14 UTC (permalink / raw)
To: Colton Lewis; +Cc: linux-perf-users
> Extend arm64/vpmu_counter_access to run all PMU counter access,
> PMCR.N validation, and overflow interrupt tests against both the
> emulated vPMU and the Partitioned PMU.
>
> Introduce enum pmu_impl (EMULATED vs. PARTITIONED) and thread it
> through the test setup helpers to configure
> KVM_ARM_VCPU_PMU_V3_ENABLE_PARTITION and
> KVM_ARM_VCPU_PMU_V3_SET_NR_COUNTERS when testing the Partitioned PMU.
>
> Exercise PMCCFILTR_EL0 and live PMCCNTR_EL0 counting before validating
> unimplemented counters so that Partitioned PMU vCPUs transition to
> VCPU_PMU_ACCESS_GUEST_OWNED even when PMCR_EL0.N == 0. On hosts without
> FEAT_HPMN0, allow n == 0 whenever max_counters >= 2 and verify that
> counter 0 (hidden behind traps while physical MDCR_EL2.HPMN == 1) still
> raises UNDEFINED on direct and indirect accesses and is RAZ/WI in the
> bitmap and software-increment registers.
>
> Skip out-of-bounds counter access exception tests for other counters
> when the PMU is partitioned, as hardware does not guarantee an UNDEFINED
> exception on accesses to counters >= MDCR_EL2.HPMN.
>
> Signed-off-by: Colton Lewis <coltonlewis@google.com>
Sashiko has reviewed this patch and found no issues.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261119185215.632334-1-coltonlewis@google.com?part=33
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v10 18/33] KVM: arm64: Enforce PMU event filter at vcpu_load()
2026-11-19 18:52 ` [PATCH v10 18/33] KVM: arm64: Enforce PMU event filter at vcpu_load() Colton Lewis
@ 2026-11-19 19:14 ` sashiko-bot
0 siblings, 0 replies; 67+ messages in thread
From: sashiko-bot @ 2026-11-19 19:14 UTC (permalink / raw)
To: Colton Lewis; +Cc: linux-perf-users
> The KVM PMU event filter uAPI requires disallowed events not to count.
> Because userspace can install or update the event filter independently
> of guest register writes, re-apply the filter on every vCPU load as well
> as on trapped PMEVTYPER<n>_EL0 and PMCCFILTR_EL0 writes, excluding all
> exception levels in hardware whenever the selected event is blocked.
>
> When running in a nested hypervisor context (is_hyp_ctxt(vcpu)), L1
> vEL2 executes at physical NS-EL1: remap guest ARMV8_PMU_INCLUDE_EL2 onto
> physical EL1 (clearing ARMV8_PMU_EXCLUDE_NS_EL1 so P == NSK) and honor
> guest MDCR_EL2.HPMD for event counters 0..HPMN-1 and MDCR_EL2.HCCD for
> PMCCNTR_EL0.
>
> Signed-off-by: Colton Lewis <coltonlewis@google.com>
Sashiko has reviewed this patch and found no issues.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261119185215.632334-1-coltonlewis@google.com?part=18
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v10 25/33] KVM: arm64: Detect overflows for the Partitioned PMU
2026-11-19 18:52 ` [PATCH v10 25/33] KVM: arm64: Detect overflows for the Partitioned PMU Colton Lewis
@ 2026-11-19 19:15 ` sashiko-bot
0 siblings, 0 replies; 67+ messages in thread
From: sashiko-bot @ 2026-11-19 19:15 UTC (permalink / raw)
To: Colton Lewis; +Cc: linux-perf-users
> Evaluate whether any enabled and unmasked guest counter has overflowed
> in kvm_pmu_part_overflow_status() so KVM updates the virtual PMU
> interrupt level in the vGIC (or userspace IRQ state) across VM entries
> and host PMU overflow interrupts:
> - Harvest any live guest overflow bits from physical pmovsset_el0 into
> shadow PMOVSSET_EL0 and clear them in hardware when the partitioned
> PMU is loaded on the local CPU.
> - Check (PMOVSSET_EL0 & PMINTENSET_EL1) against PMCR_EL0.E for guest
> counters 0..HPMN-1 and PMCCNTR_EL0, and against MDCR_EL2.HPME for
> nested hypervisor counters HPMN..N-1.
> - In kvm_pmu_handle_guest_irq(), raise KVM_REQ_IRQ_PENDING and kick the
> vCPU (or queue overflow_work when running in NMI context) when an
> unmasked guest overflow occurs.
>
> Signed-off-by: Colton Lewis <coltonlewis@google.com>
Sashiko has reviewed this patch and found no issues.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261119185215.632334-1-coltonlewis@google.com?part=25
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v10 29/33] KVM: arm64: Forward FGT traps for PMXEVTYPER_EL0 and PMXEVCNTR_EL0
2026-11-19 18:52 ` [PATCH v10 29/33] KVM: arm64: Forward FGT traps for PMXEVTYPER_EL0 and PMXEVCNTR_EL0 Colton Lewis
@ 2026-11-19 19:15 ` sashiko-bot
0 siblings, 0 replies; 67+ messages in thread
From: sashiko-bot @ 2026-11-19 19:15 UTC (permalink / raw)
To: Colton Lewis; +Cc: linux-perf-users
> When an L1 hypervisor runs a nested L2 guest with a Partitioned PMU
> (vMDCR_EL2.TPM == 0), L1 relies on fine-grained traps in HDFGRTR_EL2 and
> HDFGWTR_EL2 (PMEVTYPERn_EL0, PMCCFILTR_EL0, and PMEVCNTRn_EL0) rather
> than coarse-grained MDCR_EL2.TPM traps.
>
> Add SYS_PMXEVTYPER_EL0 and SYS_PMXEVCNTR_EL0 to encoding_to_fgt[] in
> arch/arm64/kvm/emulate-nested.c so L0 forwards fine-grained traps on
> selector-indexed PMU accesses to L1, using PMSELR_SEL to check
> HDFG{R,W}TR_EL2.PMCCFILTR_EL0 when PMSELR_EL0.SEL == 31 and
> HDFG{R,W}TR_EL2.PMEVTYPERn_EL0 otherwise.
>
> Signed-off-by: Colton Lewis <coltonlewis@google.com>
Sashiko has reviewed this patch and found no issues.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261119185215.632334-1-coltonlewis@google.com?part=29
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v10 26/33] KVM: arm64: Multiplex PMCR_EL0.E and MDCR_EL2.HPME for sub-partitioned L1
2026-11-19 18:52 ` [PATCH v10 26/33] KVM: arm64: Multiplex PMCR_EL0.E and MDCR_EL2.HPME for sub-partitioned L1 Colton Lewis
@ 2026-11-19 19:16 ` sashiko-bot
0 siblings, 0 replies; 67+ messages in thread
From: sashiko-bot @ 2026-11-19 19:16 UTC (permalink / raw)
To: Colton Lewis; +Cc: linux-perf-users
> When an L1 hypervisor sub-partitions its counters (vHPMN < N_L1, so
> kvm_pmu_hyp_counter_mask(vcpu) is non-zero), all N_L1 guest counters lie
> below physical MDCR_EL2.HPMN == N_L1 and are gated in hardware by
> physical PMCR_EL0.E. Architecturally, however, counters 0..vHPMN-1 and
> PMCCNTR_EL0 are gated by vPMCR_EL0.E while L1's hypervisor counters
> vHPMN..N_L1-1 are gated independently by vMDCR_EL2.HPME.
>
> Multiplex both domain enables onto physical PMCNTENSET_EL0 and
> PMCR_EL0.E whenever L1 sub-partitions its counters (leaving
> non-sub-partitioned guests completely untrapped):
> - Keep MDCR_EL2.TPMCR and HDFG{R,W}TR_EL2.PMCNTEN enabled when
> kvm_pmu_hyp_counter_mask(vcpu) is non-zero.
> - Maintain architectural guest PMCR_EL0 and PMCNTENSET_EL0 values in
> sys_regs[], set physical PMCR_EL0.E when either vPMCR_EL0.E or
> vMDCR_EL2.HPME is set, and program physical PMCNTENSET_EL0 with only
> those guest-enabled counters whose domain enable is active.
>
> Signed-off-by: Colton Lewis <coltonlewis@google.com>
Sashiko has reviewed this patch and found no issues.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261119185215.632334-1-coltonlewis@google.com?part=26
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v10 17/33] KVM: arm64: Context swap Partitioned PMU guest registers
2026-11-19 18:51 ` [PATCH v10 17/33] KVM: arm64: Context swap Partitioned PMU guest registers Colton Lewis
@ 2026-11-19 19:17 ` sashiko-bot
0 siblings, 0 replies; 67+ messages in thread
From: sashiko-bot @ 2026-11-19 19:17 UTC (permalink / raw)
To: Colton Lewis; +Cc: linux-perf-users
> Save and restore the guest PMU registers that are untrapped when the PMU
> is partitioned:
> - PMEVCNTRn_EL0
> - PMCCNTR_EL0
> - PMSELR_EL0 (when MDCR_EL2.TPM is clear)
> - PMUSERENR_EL0 (when MDCR_EL2.TPM is clear)
> - PMCR_EL0
> - PMCNTENSET_EL0 / PMCNTENCLR_EL0
> - PMINTENSET_EL1 / PMINTENCLR_EL1
>
> Mask counter bitmaps to guest-reserved counters so host-owned counters
> in HPMN..N-1 are left untouched. Clear hardware guest overflow flags on
> load before enabling PMINTENSET_EL1 to avoid spurious interrupts, flush
> pending hardware guest overflow flags into shadow PMOVSSET_EL0 on put,
> and restore physical PMCR_EL0.E from MDCR_EL2.HPME on put so active host
> perf events continue counting.
>
> PMEVTYPER<n>_EL0 and PMCCFILTR_EL0 are loaded in the next patch after
> applying the KVM PMU event filter. A subsequent patch makes these swaps
> lazy until the guest enables the PMU.
>
> Signed-off-by: Colton Lewis <coltonlewis@google.com>
Sashiko has reviewed this patch and found no issues.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261119185215.632334-1-coltonlewis@google.com?part=17
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v10 30/33] KVM: arm64: Support zero event counters on Partitioned PMU without HPMN0
2026-11-19 18:52 ` [PATCH v10 30/33] KVM: arm64: Support zero event counters on Partitioned PMU without HPMN0 Colton Lewis
@ 2026-11-19 19:20 ` sashiko-bot
0 siblings, 0 replies; 67+ messages in thread
From: sashiko-bot @ 2026-11-19 19:20 UTC (permalink / raw)
To: Colton Lewis; +Cc: linux-perf-users
> Programming MDCR_EL2.HPMN to 0 is CONSTRAINED UNPREDICTABLE on CPUs that
> do not implement FEAT_HPMN0, preventing userspace from configuring a
> cycle-counter-only Partitioned PMU guest (nr_pmu_counters == 0) on those
> platforms if HPMN is set directly from nr_pmu_counters.
>
> Support nr_pmu_counters == 0 without FEAT_HPMN0 whenever the PMU has at
> least two general-purpose counters by setting physical MDCR_EL2.HPMN = 1
> (via kvm_arm_pmu_part_hpmn()), reserving and quiescing physical counter 0
> alongside PMCCNTR_EL0 on vCPU load so host perf uses counters 1..N-1,
> keeping MDCR_EL2.TPMCR set so trapped PMCR_EL0 reads report N == 0, and
> enabling fine-grained traps on PMEVCNTRn_EL0, PMCNTEN, PMINTEN, and
> PMSWINC_EL0 so counter 0 remains completely hidden from the guest while
> PMCCNTR_EL0 accesses stay untrapped on FEAT_FGT hosts.
>
> Signed-off-by: Colton Lewis <coltonlewis@google.com>
Sashiko has reviewed this patch and found no issues.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261119185215.632334-1-coltonlewis@google.com?part=30
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v10 16/33] KVM: arm64: Set up MDCR_EL2 to handle a Partitioned PMU
2026-11-19 18:51 ` [PATCH v10 16/33] KVM: arm64: Set up MDCR_EL2 to handle a Partitioned PMU Colton Lewis
@ 2026-11-19 19:21 ` sashiko-bot
0 siblings, 0 replies; 67+ messages in thread
From: sashiko-bot @ 2026-11-19 19:21 UTC (permalink / raw)
To: Colton Lewis; +Cc: linux-perf-users
> Configure MDCR_EL2 for a Partitioned PMU in kvm_arm_setup_mdcr_el2():
> set HPME, HPMD, and HCCD (plus HLP on FEAT_PMUv3p5 so host counters
> retain 64-bit overflow width), program HPMN to the guest's general
> purpose counter count, and clear coarse-grained TPM and TPMCR traps when
> FEAT_FGT is available. Protect MDCR_EL2 updates with local_daif_save()
> so GICv3 pseudo-NMI PMU handlers toggling MDCR_EL2.HPME cannot race with
> kvm_arm_setup_mdcr_el2().
>
> Update kvm_nested_setup_mdcr_el2() for the two nested execution contexts:
> - Nested guest context (is_nested_ctxt(vcpu), L2 at vEL1/vEL0):
> NV_MDCR_GUEST_INCLUDE already propagates L1's virtual TPM and TPMCR.
> When L1 sub-partitions its counters (vHPMN < nr_pmu_counters, so
> kvm_pmu_hyp_counter_mask(vcpu) is non-zero), physical MDCR_EL2.HPMN
> still equals nr_pmu_counters, so hardware alone cannot stop L2 from
> accessing L1's hypervisor counters (vHPMN..nr_pmu_counters-1) or
> reading nr_pmu_counters from PMCR_EL0.N. Force TPM and TPMCR so L0
> emulates 0..vHPMN-1 locally and forwards hypervisor counter traps to
> L1.
> - Nested hypervisor context (else branch, L1): under VHE, both L1 vEL2
> (physical EL1) and VHE host EL0 (physical EL0) run in this context
> without exiting to L0 across ERET and SVC. When L1's PMUSERENR_EL0 is
> 0, hardware traps EL0 PMU accesses directly to L1 vEL2, allowing TPM
> and TPMCR to stay clear for untrapped L1 vEL2 access. When L1 enables
> EL0 access in PMUSERENR_EL0, propagate L1's virtual TPM/TPMCR (and set
> TPM/TPMCR if L1 sub-partitions counters) so VHE host EL0 cannot bypass
> L1's traps or vHPMN boundary.
>
> Signed-off-by: Colton Lewis <coltonlewis@google.com>
Sashiko has reviewed this patch and found no issues.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261119185215.632334-1-coltonlewis@google.com?part=16
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v10 22/33] KVM: arm64: Apply dynamic guest counter reservations
2026-11-19 18:52 ` [PATCH v10 22/33] KVM: arm64: Apply dynamic guest counter reservations Colton Lewis
@ 2026-11-19 19:21 ` sashiko-bot
0 siblings, 0 replies; 67+ messages in thread
From: sashiko-bot @ 2026-11-19 19:21 UTC (permalink / raw)
To: Colton Lewis; +Cc: linux-perf-users
> Reserve and release guest PMU counters dynamically during vCPU load and
> put rather than statically at VM creation.
>
> Add kvm_pmu_set_guest_counters() in arch/arm64/kvm/pmu-direct.c, called
> from __kvm_pmu_load() and __kvm_pmu_put() with local interrupts
> disabled. When the requested guest counter mask collides with active
> host events in cpuc->used_mask (or when releasing counters after
> squeezing a host event), invoke perf_pmu_resched_update() with
> kvm_pmu_update_mask() to update per-CPU cpuc->cntr_mask between
> scheduling host events out and back in; otherwise update cpuc->cntr_mask
> directly. Warn once in armpmu_add() if a pinned host event fails counter
> allocation while guest counters are reserved.
>
> Skip releasing and re-reserving host counters across nested ERET and
> nested exception transitions (IN_NESTED_ERET and IN_NESTED_EXCEPTION)
> since the vCPU remains on the same physical CPU.
>
> Signed-off-by: Colton Lewis <coltonlewis@google.com>
Sashiko has reviewed this patch and found no issues.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261119185215.632334-1-coltonlewis@google.com?part=22
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v10 14/33] KVM: arm64: Add Partitioned PMU register trap handlers
2026-11-19 18:51 ` [PATCH v10 14/33] KVM: arm64: Add Partitioned PMU register trap handlers Colton Lewis
@ 2026-11-19 19:21 ` sashiko-bot
0 siblings, 0 replies; 67+ messages in thread
From: sashiko-bot @ 2026-11-19 19:21 UTC (permalink / raw)
To: Colton Lewis; +Cc: kvmarm, kvm, linux-perf-users, Oliver Upton, Marc Zyngier
Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Low] KVM: arm64: missing 32-bit counter mask in
kvm_pmu_direct_cntr_read()
--
commit a61557b3819ee55851d36163d0d9444fd60c695d
Author: Colton Lewis <coltonlewis@google.com>
KVM: arm64: Add Partitioned PMU register trap handlers
This commit adds direct PMU register read and write handlers for VMs using
a Partitioned PMU and dispatches trapped PMU accesses to them.
> diff --git a/arch/arm64/kvm/pmu-direct.c b/arch/arm64/kvm/pmu-direct.c
> index b1c180ae79d9d..13f5bcc2a9084 100644
> --- a/arch/arm64/kvm/pmu-direct.c
> +++ b/arch/arm64/kvm/pmu-direct.c
[ ... ]
> @@ -79,6 +92,245 @@ static u64 kvm_vcpu_pmu_guest_counter_mask(struct kvm_vcpu *vcpu)
> return 0;
> }
[ ... ]
> +static u64 kvm_pmu_direct_cntr_read(struct kvm_vcpu *vcpu, u8 idx)
> +{
> + lockdep_assert_preemption_disabled();
> +
> + if (!kvm_pmu_part_on_cpu(vcpu))
> + return __vcpu_sys_reg(vcpu, PMEVCNTR0_EL0 + idx);
> +
> + if (idx == ARMV8_PMU_CYCLE_IDX)
> + return read_pmccntr();
> +
> + return read_pmevcntrn(idx);
> +}
> +
> +static void kvm_pmu_direct_cntr_write(struct kvm_vcpu *vcpu, u8 idx, u64 val)
> +{
> + lockdep_assert_preemption_disabled();
> +
> + __vcpu_assign_sys_reg(vcpu, PMEVCNTR0_EL0 + idx, val);
> + if (!kvm_pmu_part_on_cpu(vcpu))
> + return;
> +
> + if (idx == ARMV8_PMU_CYCLE_IDX)
> + write_pmccntr(val);
> + else
> + write_pmevcntrn(idx, val);
> +}
[ ... ]
> +void kvm_pmu_direct_reg_write(struct kvm_vcpu *vcpu, unsigned int reg,
> + u64 val, bool set)
> +{
> + unsigned long flags;
> + u64 mask;
> + int idx;
> +
> + guard(preempt)();
> +
> + switch (reg) {
> + case PMCR_EL0:
> + kvm_pmu_direct_pmcr_write(vcpu, val);
> + break;
> + case PMSELR_EL0:
> + if (kvm_pmu_part_on_cpu(vcpu) &&
> + !(vcpu->arch.mdcr_el2 & MDCR_EL2_TPM))
> + write_sysreg(val, pmselr_el0);
> + __vcpu_assign_sys_reg(vcpu, reg, val);
> + break;
> + case PMEVCNTR0_EL0 ... PMCCNTR_EL0:
> + idx = reg - PMEVCNTR0_EL0;
> + if (vcpu_mode_is_32bit(vcpu) && idx != ARMV8_PMU_CYCLE_IDX)
> + val = (kvm_pmu_direct_cntr_read(vcpu, idx) &
> + GENMASK_ULL(63, 32)) |
> + lower_32_bits(val);
> + kvm_pmu_direct_cntr_write(vcpu, idx, val);
> + break;
[Severity: Low]
Should kvm_pmu_direct_cntr_read() and kvm_pmu_direct_reg_write() mask
general-purpose event counters to 32 bits when the VM does not have
FEAT_PMUv3p5?
kvm_pmu_direct_pmcr_read() and kvm_pmu_direct_pmcr_write() clear
ARMV8_PMU_PMCR_LP when !kvm_has_feat(vcpu->kvm, ID_AA64DFR0_EL1, PMUVer,
V3P5), and the emulated path in kvm_pmu_get_pmc_value() truncates
general-purpose counters via lower_32_bits() when FEAT_PMUv3p5 is absent.
Here, trapped 64-bit guest accesses to PMEVCNTR<n>_EL0 or PMXEVCNTR_EL0
store and return the full 64-bit value, exposing bits [63:32] to a guest
configured without FEAT_PMUv3p5.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261119185215.632334-1-coltonlewis@google.com?part=14
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v10 24/33] perf: arm_pmuv3: Handle IRQs for Partitioned PMU guest counters
2026-11-19 18:52 ` [PATCH v10 24/33] perf: arm_pmuv3: Handle IRQs for Partitioned PMU guest counters Colton Lewis
@ 2026-11-19 19:22 ` sashiko-bot
0 siblings, 0 replies; 67+ messages in thread
From: sashiko-bot @ 2026-11-19 19:22 UTC (permalink / raw)
To: Colton Lewis; +Cc: linux-perf-users
> Because ARM hardware does not directly inject PMU PPIs into guests,
> overflows on partitioned guest counters raise physical PMU interrupts
> that enter the host's armv8pmu_handle_irq() handler.
>
> Update armv8pmu_handle_irq() to read PMOVSCLR_EL0 without blindly
> clearing guest overflow bits:
> - Pass pmovsr to kvm_pmu_handle_guest_irq(), which clears guest-reserved
> overflow bits in hardware and latches them into the running vCPU's
> shadow PMOVSSET_EL0 register.
> - Mask out guest-reserved counter bits before clearing and processing
> host perf overflows, returning IRQ_HANDLED early when only guest
> counters overflowed or no host events are active.
>
> Also add kvm_pmu_host_start() and kvm_pmu_host_stop() so
> armv8pmu_start() and armv8pmu_stop() toggle MDCR_EL2.HPME rather than
> PMCR_EL0.E while a partitioned guest's PMU state is loaded on the CPU
> (leaving guest counters 0..HPMN-1 and PMCCNTR_EL0 under guest PMCR_EL0.E
> control), guarding local_daif_save() with !in_nmi() for GICv3 pseudo-NMI
> interrupt handlers.
>
> Signed-off-by: Colton Lewis <coltonlewis@google.com>
Sashiko has reviewed this patch and found no issues.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261119185215.632334-1-coltonlewis@google.com?part=24
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v10 23/33] KVM: arm64: Implement lazy PMU context swaps
2026-11-19 18:52 ` [PATCH v10 23/33] KVM: arm64: Implement lazy PMU context swaps Colton Lewis
@ 2026-11-19 19:24 ` sashiko-bot
0 siblings, 0 replies; 67+ messages in thread
From: sashiko-bot @ 2026-11-19 19:24 UTC (permalink / raw)
To: Colton Lewis; +Cc: linux-perf-users
> Since many guests never use the PMU, avoid reserving hardware counters
> and context swapping PMU registers on every vCPU load and put until the
> guest actively enables counting or overflow interrupts.
>
> Track per-vCPU PMU ownership via enum vcpu_pmu_register_access
> (VCPU_PMU_ACCESS_FREE vs. VCPU_PMU_ACCESS_GUEST_OWNED):
> - While FREE, keep MDCR_EL2.HPMN at the full host counter count with TPM
> and TPMCR set so host perf retains all counters including PMCCNTR_EL0.
> Trapped reads and non-enabling writes (such as guest boot PMU probes)
> operate on shadow sys_regs[] without claiming hardware counters.
> - When the guest enables counting or interrupts (setting PMCR_EL0.E,
> enabling MDCR_EL2.HPME with hypervisor counters, or setting guest bits
> in PMCNTENSET_EL0 or PMINTENSET_EL1), transition to GUEST_OWNED via
> kvm_pmu_set_guest_owned(), reserve the guest counter partition, and
> load guest state into hardware.
>
> Add kvm_pmu_reload_part() to service KVM_REQ_RELOAD_PMU for partitioned
> VMs (triggered by MDCR_EL2 or PMUSERENR_EL0 trap changes, vCPU reset,
> and KVM_SET_ONE_REG writes). Unlike the emulated PMU, which only masks
> shadow registers and reprograms perf_events, a partitioned PMU loaded on
> the CPU must flush live hardware registers to sys_regs[], mask
> unimplemented counter bits, recompute FGT, MDCR_EL2, and hardware event
> filters, and reload hardware without dropping its host counter
> reservation. When not yet loaded (such as after vCPU reset or
> KVM_SET_ONE_REG), kvm_pmu_reload_part() transitions to GUEST_OWNED only
> if PMCR_EL0.E or MDCR_EL2.HPME is enabled, avoiding false activation
> from reset_unknown() poison bits in PMCNTENSET_EL0 or PMINTENSET_EL1.
>
> Signed-off-by: Colton Lewis <coltonlewis@google.com>
Sashiko has reviewed this patch and found no issues.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261119185215.632334-1-coltonlewis@google.com?part=23
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v10 27/33] KVM: arm64: Emulate mismatched LP and HLP overflow widths for sub-partitioned L1
2026-11-19 18:52 ` [PATCH v10 27/33] KVM: arm64: Emulate mismatched LP and HLP overflow widths " Colton Lewis
@ 2026-11-19 19:26 ` sashiko-bot
0 siblings, 0 replies; 67+ messages in thread
From: sashiko-bot @ 2026-11-19 19:26 UTC (permalink / raw)
To: Colton Lewis; +Cc: kvmarm, kvm, linux-perf-users, Oliver Upton, Marc Zyngier
Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Medium] KVM: arm64: broken PMU counter chaining for ovf32_mask
counters
--
commit 210ea84a824c63596df96f6f9ef557724dabffcb
Author: Colton Lewis <coltonlewis@google.com>
KVM: arm64: Emulate mismatched LP and HLP overflow widths for sub-partitioned L1
This commit emulates mismatched 32-bit and 64-bit counter overflow widths
(vPMCR_EL0.LP != vMDCR_EL2.HLP) when an L1 guest sub-partitions a
Partitioned PMU by forcing physical PMCR_EL0.LP to 1, biasing the 32-bit
counter sub-range by 2^32 events below 64-bit overflow, and trapping
PMEVCNTR<n>_EL0 accesses to translate counter values.
> diff --git a/arch/arm64/kvm/pmu-direct.c b/arch/arm64/kvm/pmu-direct.c
> index 725b5fc17533e..b7bec06b36b0a 100644
> --- a/arch/arm64/kvm/pmu-direct.c
> +++ b/arch/arm64/kvm/pmu-direct.c
[ ... ]
> @@ -123,12 +160,109 @@ static void kvm_pmu_part_sync_hw_enables(struct kvm_vcpu *vcpu)
> pmcr &= pmcr_mask;
> if (mdcr & MDCR_EL2_HPME)
> pmcr |= ARMV8_PMU_PMCR_E;
> + if (kvm_has_feat(vcpu->kvm, ID_AA64DFR0_EL1, PMUVer, V3P5)) {
> + if (ev_mask && !(ev_mask & ~hyp_mask)) {
> + if (mdcr & MDCR_EL2_HLP)
> + pmcr |= ARMV8_PMU_PMCR_LP;
> + else
> + pmcr &= ~ARMV8_PMU_PMCR_LP;
> + } else if (vcpu->arch.pmu.ovf32_mask) {
> + pmcr |= ARMV8_PMU_PMCR_LP;
> + }
> + }
[Severity: Medium]
Does forcing physical PMCR_EL0.LP to 1 break counter chaining for counters
in ovf32_mask?
Counters in ovf32_mask are architecturally configured for 32-bit overflow,
where an even counter overflowing at 32 bits increments an adjacent odd
counter programmed with ARMV8_PMUV3_PERFCTR_CHAIN (compare
kvm_pmu_counter_can_chain() in arch/arm64/kvm/pmu-emul.c). Because all
guest counters sit below physical MDCR_EL2.HPMN, setting physical
PMCR_EL0.LP to 1 puts the 32-bit sub-range into 64-bit overflow mode in
hardware, which disables hardware chaining, and
kvm_pmu_part_rebias_single() does not propagate the 32-bit overflow to a
chained odd counter in software.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261119185215.632334-1-coltonlewis@google.com?part=27
^ permalink raw reply [flat|nested] 67+ messages in thread
end of thread, other threads:[~2026-11-19 19:26 UTC | newest]
Thread overview: 67+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-19 18:51 [PATCH v10 00/33] ARM64 PMU Partitioning Colton Lewis
2026-11-19 18:51 ` [PATCH v10 01/33] arm64: cpufeature: Add cpucap for HPMN0 Colton Lewis
2026-11-19 19:08 ` sashiko-bot
2026-11-19 18:51 ` [PATCH v10 02/33] KVM: arm64: Reorganize PMU includes Colton Lewis
2026-11-19 19:00 ` sashiko-bot
2026-11-19 18:51 ` [PATCH v10 03/33] KVM: arm64: Reorganize PMU functions Colton Lewis
2026-11-19 18:57 ` sashiko-bot
2026-11-19 18:51 ` [PATCH v10 04/33] KVM: arm64: Use nr_pmu_counters in kvm_pmu_implemented_counter_mask() Colton Lewis
2026-11-19 19:08 ` sashiko-bot
2026-11-19 18:51 ` [PATCH v10 05/33] KVM: arm64: Ignore HPMN for userspace PMU register access Colton Lewis
2026-11-19 18:58 ` sashiko-bot
2026-11-19 18:51 ` [PATCH v10 06/33] perf: arm_pmuv3: Generalize counter bitmasks Colton Lewis
2026-11-19 18:56 ` sashiko-bot
2026-11-19 18:51 ` [PATCH v10 07/33] perf: arm_pmuv3: Move counter allocation mask to per-CPU struct pmu_hw_events Colton Lewis
2026-11-19 19:04 ` sashiko-bot
2026-11-19 18:51 ` [PATCH v10 08/33] perf: arm_pmuv3: Check cntr_mask before using pmccntr Colton Lewis
2026-11-19 19:01 ` sashiko-bot
2026-11-19 18:51 ` [PATCH v10 09/33] perf: arm_pmuv3: Allocate counter indices from high to low Colton Lewis
2026-11-19 18:57 ` sashiko-bot
2026-11-19 18:51 ` [PATCH v10 10/33] perf: arm_pmuv3: Expose number of general-purpose counters in sysfs Colton Lewis
2026-11-19 18:57 ` sashiko-bot
2026-11-19 18:51 ` [PATCH v10 11/33] KVM: arm64: Add initial scaffolding for Partitioned PMU Colton Lewis
2026-11-19 18:58 ` sashiko-bot
2026-11-19 18:51 ` [PATCH v10 12/33] KVM: arm64: Set up FGT " Colton Lewis
2026-11-19 19:13 ` sashiko-bot
2026-11-19 18:51 ` [PATCH v10 13/33] KVM: arm64: Make __vcpu_rmw_sys_reg() atomic Colton Lewis
2026-11-19 18:58 ` sashiko-bot
2026-11-19 18:51 ` [PATCH v10 14/33] KVM: arm64: Add Partitioned PMU register trap handlers Colton Lewis
2026-11-19 19:21 ` sashiko-bot
2026-11-19 18:51 ` [PATCH v10 15/33] KVM: arm64: Construct mdcr_el2 in a local variable in kvm_arm_setup_mdcr_el2() Colton Lewis
2026-11-19 18:56 ` sashiko-bot
2026-11-19 18:51 ` [PATCH v10 16/33] KVM: arm64: Set up MDCR_EL2 to handle a Partitioned PMU Colton Lewis
2026-11-19 19:21 ` sashiko-bot
2026-11-19 18:51 ` [PATCH v10 17/33] KVM: arm64: Context swap Partitioned PMU guest registers Colton Lewis
2026-11-19 19:17 ` sashiko-bot
2026-11-19 18:52 ` [PATCH v10 18/33] KVM: arm64: Enforce PMU event filter at vcpu_load() Colton Lewis
2026-11-19 19:14 ` sashiko-bot
2026-11-19 18:52 ` [PATCH v10 19/33] perf: Add perf_pmu_resched_update() Colton Lewis
2026-11-19 19:02 ` sashiko-bot
2026-11-19 18:52 ` [PATCH v10 20/33] perf/core: Prioritize kernel-owned pinned events when scheduling groups Colton Lewis
2026-11-19 19:01 ` sashiko-bot
2026-11-19 18:52 ` [PATCH v10 21/33] KVM: arm64: Allow kvm_vcpu_pmu_resync_el0() to resync filters in process context Colton Lewis
2026-11-19 19:07 ` sashiko-bot
2026-11-19 18:52 ` [PATCH v10 22/33] KVM: arm64: Apply dynamic guest counter reservations Colton Lewis
2026-11-19 19:21 ` sashiko-bot
2026-11-19 18:52 ` [PATCH v10 23/33] KVM: arm64: Implement lazy PMU context swaps Colton Lewis
2026-11-19 19:24 ` sashiko-bot
2026-11-19 18:52 ` [PATCH v10 24/33] perf: arm_pmuv3: Handle IRQs for Partitioned PMU guest counters Colton Lewis
2026-11-19 19:22 ` sashiko-bot
2026-11-19 18:52 ` [PATCH v10 25/33] KVM: arm64: Detect overflows for the Partitioned PMU Colton Lewis
2026-11-19 19:15 ` sashiko-bot
2026-11-19 18:52 ` [PATCH v10 26/33] KVM: arm64: Multiplex PMCR_EL0.E and MDCR_EL2.HPME for sub-partitioned L1 Colton Lewis
2026-11-19 19:16 ` sashiko-bot
2026-11-19 18:52 ` [PATCH v10 27/33] KVM: arm64: Emulate mismatched LP and HLP overflow widths " Colton Lewis
2026-11-19 19:26 ` sashiko-bot
2026-11-19 18:52 ` [PATCH v10 28/33] KVM: arm64: Emulate trapped PMSWINC_EL0 writes for Partitioned PMU Colton Lewis
2026-11-19 19:13 ` sashiko-bot
2026-11-19 18:52 ` [PATCH v10 29/33] KVM: arm64: Forward FGT traps for PMXEVTYPER_EL0 and PMXEVCNTR_EL0 Colton Lewis
2026-11-19 19:15 ` sashiko-bot
2026-11-19 18:52 ` [PATCH v10 30/33] KVM: arm64: Support zero event counters on Partitioned PMU without HPMN0 Colton Lewis
2026-11-19 19:20 ` sashiko-bot
2026-11-19 18:52 ` [PATCH v10 31/33] KVM: arm64: Add vCPU device attr to partition the PMU Colton Lewis
2026-11-19 19:14 ` sashiko-bot
2026-11-19 18:52 ` [PATCH v10 32/33] KVM: selftests: Add find_bit to KVM library Colton Lewis
2026-11-19 19:07 ` sashiko-bot
2026-11-19 18:52 ` [PATCH v10 33/33] KVM: arm64: selftests: Add test case for Partitioned PMU Colton Lewis
2026-11-19 19:14 ` sashiko-bot
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox