[PATCH 0/8] KVM: Add support for hardware-switched mediated PMU

[Sandipan Das <sandipan.das@amd.com>]

Mediated PMU currently saves and restores the guest PMU state in
software on every world switch. On Zen 5 and later processors, it is
possible to offload this to the hardware, which automatically saves and
restores the guest counter state to and from a vendor save area (VMCB)
during world switches. This feature is called PMC virtualization and
detailed information about it can be found in Volume 2 of the AMD64
Architecture Programmer's Manual under Section 15.39.

This series extends the mediated PMU framework with a hardware-switched
mode and adds the AMD (SVM) implementation, including nested support.

Performance
-----------
The difference in the world switch overhead between software-switched
and hardware-switched modes is measured with the x86/vmexit test from
kvm-unit-tests on a system with an AMD EPYC 9755 processor. The values
are in cycles per operation (lower is better), averaged over 5 runs.

------------------------------------------------------------------------
test                               sw-switched  hw-switched         gain
------------------------------------------------------------------------
cpuid                                     5333         2592       +51.4%
vmcall                                   15858         7619       +52.0%
wbinvd                                    5256         2517       +52.1%
invd                                      1857         2147       -15.6%
mov_from_cr8                                18           18         0.0%
mov_to_cr8                                 157          156        +0.6%
inl_from_pmtimer                         25934        21176       +18.3%
inl_from_qemu                            25881        21156       +18.3%
inl_from_kernel                           5521         2772       +49.8%
outl_to_kernel                            5522         2778       +49.7%
mov_dr                                     114          114         0.0%
self_ipi_sti_nop                          1020         1015        +0.5%
  ipi self_ipi_sti_nop                     310          307        +1.0%
  eoi self_ipi_sti_nop                     242          241        +0.4%
self_ipi_sti_hlt                          2714         2704        +0.4%
  ipi self_ipi_sti_hlt                     310          307        +1.0%
  eoi self_ipi_sti_hlt                     242          240        +0.8%
self_ipi_tpr                              1272         1263        +0.7%
  ipi self_ipi_tpr                         309          307        +0.6%
  eoi self_ipi_tpr                         241          240        +0.4%
self_ipi_tpr_sti_nop                      1304         1295        +0.7%
  ipi self_ipi_tpr_sti_nop                 309          307        +0.6%
  eoi self_ipi_tpr_sti_nop                 242          241        +0.4%
self_ipi_tpr_sti_hlt                      3003         2991        +0.4%
  ipi self_ipi_tpr_sti_hlt                 309          307        +0.6%
  eoi self_ipi_tpr_sti_hlt                 242          240        +0.8%
x2apic_self_ipi_sti_nop                    899          895        +0.4%
  ipi x2apic_self_ipi_sti_nop              191          190        +0.5%
  eoi x2apic_self_ipi_sti_nop              242          241        +0.4%
x2apic_self_ipi_sti_hlt                   2592         2587        +0.2%
  ipi x2apic_self_ipi_sti_hlt              191          190        +0.5%
  eoi x2apic_self_ipi_sti_hlt              242          241        +0.4%
x2apic_self_ipi_tpr                       1159         1151        +0.7%
  ipi x2apic_self_ipi_tpr                  193          193         0.0%
  eoi x2apic_self_ipi_tpr                  241          240        +0.4%
x2apic_self_ipi_tpr_sti_nop               1189         1180        +0.8%
  ipi x2apic_self_ipi_tpr_sti_nop          193          193         0.0%
  eoi x2apic_self_ipi_tpr_sti_nop          242          241        +0.4%
x2apic_self_ipi_tpr_sti_hlt               2887         2875        +0.4%
  ipi x2apic_self_ipi_tpr_sti_hlt          194          192        +1.0%
  eoi x2apic_self_ipi_tpr_sti_hlt          242          239        +1.2%
ple_round_robin                             19           19         0.0%
wr_kernel_gs_base                           73           73         0.0%
wr_ibpb_msr                                563          557        +1.1%
wr_tsc_adjust_msr                         5352         2603       +51.4%
rd_tsc_adjust_msr                         5330         2586       +51.5%
toggle_cr0_wp                            11319         5810       +48.7%
toggle_cr4_pge                           11743         6242       +46.8%
mmio-no-eventfd:pci-mem                  31890        26835       +15.9%
mmio-wildcard-eventfd:pci-mem             6461         3666       +43.3%
mmio-datamatch-eventfd:pci-mem            6449         3671       +43.1%
portio-no-eventfd:pci-io                 25218        21455       +14.9%
portio-wildcard-eventfd:pci-io            5512         2774       +49.7%
portio-datamatch-eventfd:pci-io           5524         2780       +49.7%
------------------------------------------------------------------------

The largest gains are for wbinvd (+52.1%), vmcall (+52.0%), and cpuid
(+51.4%), followed by the TSC-adjust MSR accesses (~+51%), the
kernel-handled port I/O paths (~+50%), and the control-register toggles
(~+47-49%). The wildcard and datamatch MMIO / port-I/O paths improve by
~+43-50%, while the slower no-eventfd MMIO / port-I/O and pmtimer/qemu
port-I/O paths improve by ~+15-18%.

The only meaningful regression is invd (-15.6%). The self-IPI and EOI
subtests move by at most 1.2% and wr_kernel_gs_base is unchanged, all
within run-to-run noise.

Design
------
  * The entity responsible for switching the guest PMU state is tracked
    per-VM via a new set of mediated PMU capability flags in kvm_arch.
    In software-switched mode the state resides in struct kvm_pmu; in
    hardware-switched mode it resides in the vendor save area.
  * Event filtering, instruction emulation, and intercepted RDPMC still
    need the state in struct kvm_pmu, so new vendor state-sync PMU ops
    synchronize specific MSRs between struct kvm_pmu and the vendor
    save area on demand.
  * On SVM, the feature depends on VNMI for guest PMI delivery and is
    controlled by the new "vpmc" kvm_amd module parameter, enabled by
    default when the host supports it and mediated PMU is enabled.

Limitations
-----------
  * Not supported for SEV-ES and SEV-SNP guests: the guest PMU state
    resides in an encrypted VMSA that is inaccessible to the event
    filtering logic.
  * With VNMI and no AVIC, overflow interrupts are always delivered as
    NMIs because the APIC is emulated and the LVTPC is ignored. This
    shows up as expected behavioral differences in the x86/pmu test of
    kvm-unit-tests.

Overview
--------
Patches 1-3 extend the mediated PMU framework with hardware-switched
mode: the capability flags, the vendor state-sync ops, and the generic
support. Patches 4-6 add the SVM enablement: the PerfCtrVirt feature
bit, the VMCB save-area fields, and the SVM implementation. Patch 7
adds nested (nSVM) support to use the feature with L2 guests. Patch 8
explicitly disables the feature for SEV-ES and SEV-SNP guests.

Tested with kvm-unit-tests (KUT) and perf fuzzer. KUT's x86/pmu test
requires patching to accept overflow interrupts as NMIs (limitation
because of VNMI ignoring the LVTPC). It is recommended to apply the
following patches before testing.
https://lore.kernel.org/kvm/3bfbf15c22652ba00cf4a16fe9e0a3bfe7071f97.1784096302.git.sandipan.das@amd.com/
https://lore.kernel.org/kvm/b075ec13e95167da1f2229626713c91d291bb75b.1784097178.git.sandipan.das@amd.com/
https://lore.kernel.org/all/106bc3a4660653cb7750f3ae07a4dbca42c2762c.1786356229.git.sandipan.das@amd.com/

base-commit: 6cf46b37b0356b3408f8ba949e10462cc71b533d
             (tag: kvm-x86-next-2026.08.07)

Previous versions can be found at:
rfc: https://lore.kernel.org/kvm/cover.1762960531.git.sandipan.das@amd.com/

Changes in v1:
  * Introduce mediated PMU capabilities for hardware PMU virtualization
    features instead of perf PMU capabilities.
  * Make VNMI a hard requirement since AVIC can be inhibited.
  * Introduce new PMU ops to access VMCB PMU state instead of relying
    on host-initiated {get,set}_msr() ops.
  * Make VMCB PMU state accessible via KVM_{GET,SET}_MSRS for migration.
  * Implement support for nested SVM.
  * Disable support for SEV-ES and SEV-SNP guests explicitly.

Sandipan Das (8):
  KVM: x86/pmu: Add mediated PMU capability flags
  KVM: x86/pmu: Add PMU ops for vendor state sync
  KVM: x86/pmu: Add support for hardware-switched PMU
  x86/cpufeatures: Add PerfCtrVirt feature bit
  KVM: SVM: Add VMCB fields for PMC virtualization
  KVM: SVM: Add support for hardware-switched PMU
  KVM: nSVM: Add support for hardware-switched PMU
  KVM: SEV: Disallow the use of hardware-switched PMU

 arch/x86/include/asm/cpufeatures.h     |   1 +
 arch/x86/include/asm/kvm-x86-pmu-ops.h |   3 +
 arch/x86/include/asm/kvm_host.h        |  11 +++
 arch/x86/include/asm/svm.h             |  13 ++-
 arch/x86/kvm/cpuid.c                   |   1 +
 arch/x86/kvm/pmu.c                     | 102 +++++++++++++++++----
 arch/x86/kvm/pmu.h                     |  29 ++++++
 arch/x86/kvm/svm/nested.c              |  45 +++++++++-
 arch/x86/kvm/svm/pmu.c                 | 117 +++++++++++++++++++++++++
 arch/x86/kvm/svm/sev.c                 |   3 +
 arch/x86/kvm/svm/svm.c                 |  58 +++++++++++-
 arch/x86/kvm/svm/svm.h                 |  19 ++++
 arch/x86/kvm/vmx/pmu_intel.c           |   2 +
 arch/x86/kvm/vmx/vmx.c                 |   3 +-
 arch/x86/kvm/x86.c                     |   2 +
 15 files changed, 384 insertions(+), 25 deletions(-)

-- 
2.53.0