[PATCH 0/8] KVM: Add support for hardware-switched mediated PMU

[PATCH 0/8] KVM: Add support for hardware-switched mediated PMU

[Sandipan Das]

Mediated PMU currently saves and restores the guest PMU state in
software on every world switch. On Zen 5 and later processors, it is
possible to offload this to the hardware, which automatically saves and
restores the guest counter state to and from a vendor save area (VMCB)
during world switches. This feature is called PMC virtualization and
detailed information about it can be found in Volume 2 of the AMD64
Architecture Programmer's Manual under Section 15.39.

This series extends the mediated PMU framework with a hardware-switched
mode and adds the AMD (SVM) implementation, including nested support.

Performance
-----------
The difference in the world switch overhead between software-switched
and hardware-switched modes is measured with the x86/vmexit test from
kvm-unit-tests on a system with an AMD EPYC 9755 processor. The values
are in cycles per operation (lower is better), averaged over 5 runs.

------------------------------------------------------------------------
test                               sw-switched  hw-switched         gain
------------------------------------------------------------------------
cpuid                                     5333         2592       +51.4%
vmcall                                   15858         7619       +52.0%
wbinvd                                    5256         2517       +52.1%
invd                                      1857         2147       -15.6%
mov_from_cr8                                18           18         0.0%
mov_to_cr8                                 157          156        +0.6%
inl_from_pmtimer                         25934        21176       +18.3%
inl_from_qemu                            25881        21156       +18.3%
inl_from_kernel                           5521         2772       +49.8%
outl_to_kernel                            5522         2778       +49.7%
mov_dr                                     114          114         0.0%
self_ipi_sti_nop                          1020         1015        +0.5%
  ipi self_ipi_sti_nop                     310          307        +1.0%
  eoi self_ipi_sti_nop                     242          241        +0.4%
self_ipi_sti_hlt                          2714         2704        +0.4%
  ipi self_ipi_sti_hlt                     310          307        +1.0%
  eoi self_ipi_sti_hlt                     242          240        +0.8%
self_ipi_tpr                              1272         1263        +0.7%
  ipi self_ipi_tpr                         309          307        +0.6%
  eoi self_ipi_tpr                         241          240        +0.4%
self_ipi_tpr_sti_nop                      1304         1295        +0.7%
  ipi self_ipi_tpr_sti_nop                 309          307        +0.6%
  eoi self_ipi_tpr_sti_nop                 242          241        +0.4%
self_ipi_tpr_sti_hlt                      3003         2991        +0.4%
  ipi self_ipi_tpr_sti_hlt                 309          307        +0.6%
  eoi self_ipi_tpr_sti_hlt                 242          240        +0.8%
x2apic_self_ipi_sti_nop                    899          895        +0.4%
  ipi x2apic_self_ipi_sti_nop              191          190        +0.5%
  eoi x2apic_self_ipi_sti_nop              242          241        +0.4%
x2apic_self_ipi_sti_hlt                   2592         2587        +0.2%
  ipi x2apic_self_ipi_sti_hlt              191          190        +0.5%
  eoi x2apic_self_ipi_sti_hlt              242          241        +0.4%
x2apic_self_ipi_tpr                       1159         1151        +0.7%
  ipi x2apic_self_ipi_tpr                  193          193         0.0%
  eoi x2apic_self_ipi_tpr                  241          240        +0.4%
x2apic_self_ipi_tpr_sti_nop               1189         1180        +0.8%
  ipi x2apic_self_ipi_tpr_sti_nop          193          193         0.0%
  eoi x2apic_self_ipi_tpr_sti_nop          242          241        +0.4%
x2apic_self_ipi_tpr_sti_hlt               2887         2875        +0.4%
  ipi x2apic_self_ipi_tpr_sti_hlt          194          192        +1.0%
  eoi x2apic_self_ipi_tpr_sti_hlt          242          239        +1.2%
ple_round_robin                             19           19         0.0%
wr_kernel_gs_base                           73           73         0.0%
wr_ibpb_msr                                563          557        +1.1%
wr_tsc_adjust_msr                         5352         2603       +51.4%
rd_tsc_adjust_msr                         5330         2586       +51.5%
toggle_cr0_wp                            11319         5810       +48.7%
toggle_cr4_pge                           11743         6242       +46.8%
mmio-no-eventfd:pci-mem                  31890        26835       +15.9%
mmio-wildcard-eventfd:pci-mem             6461         3666       +43.3%
mmio-datamatch-eventfd:pci-mem            6449         3671       +43.1%
portio-no-eventfd:pci-io                 25218        21455       +14.9%
portio-wildcard-eventfd:pci-io            5512         2774       +49.7%
portio-datamatch-eventfd:pci-io           5524         2780       +49.7%
------------------------------------------------------------------------

The largest gains are for wbinvd (+52.1%), vmcall (+52.0%), and cpuid
(+51.4%), followed by the TSC-adjust MSR accesses (~+51%), the
kernel-handled port I/O paths (~+50%), and the control-register toggles
(~+47-49%). The wildcard and datamatch MMIO / port-I/O paths improve by
~+43-50%, while the slower no-eventfd MMIO / port-I/O and pmtimer/qemu
port-I/O paths improve by ~+15-18%.

The only meaningful regression is invd (-15.6%). The self-IPI and EOI
subtests move by at most 1.2% and wr_kernel_gs_base is unchanged, all
within run-to-run noise.

Design
------
  * The entity responsible for switching the guest PMU state is tracked
    per-VM via a new set of mediated PMU capability flags in kvm_arch.
    In software-switched mode the state resides in struct kvm_pmu; in
    hardware-switched mode it resides in the vendor save area.
  * Event filtering, instruction emulation, and intercepted RDPMC still
    need the state in struct kvm_pmu, so new vendor state-sync PMU ops
    synchronize specific MSRs between struct kvm_pmu and the vendor
    save area on demand.
  * On SVM, the feature depends on VNMI for guest PMI delivery and is
    controlled by the new "vpmc" kvm_amd module parameter, enabled by
    default when the host supports it and mediated PMU is enabled.

Limitations
-----------
  * Not supported for SEV-ES and SEV-SNP guests: the guest PMU state
    resides in an encrypted VMSA that is inaccessible to the event
    filtering logic.
  * With VNMI and no AVIC, overflow interrupts are always delivered as
    NMIs because the APIC is emulated and the LVTPC is ignored. This
    shows up as expected behavioral differences in the x86/pmu test of
    kvm-unit-tests.

Overview
--------
Patches 1-3 extend the mediated PMU framework with hardware-switched
mode: the capability flags, the vendor state-sync ops, and the generic
support. Patches 4-6 add the SVM enablement: the PerfCtrVirt feature
bit, the VMCB save-area fields, and the SVM implementation. Patch 7
adds nested (nSVM) support to use the feature with L2 guests. Patch 8
explicitly disables the feature for SEV-ES and SEV-SNP guests.

Tested with kvm-unit-tests (KUT) and perf fuzzer. KUT's x86/pmu test
requires patching to accept overflow interrupts as NMIs (limitation
because of VNMI ignoring the LVTPC). It is recommended to apply the
following patches before testing.
https://lore.kernel.org/kvm/3bfbf15c22652ba00cf4a16fe9e0a3bfe7071f97.1784096302.git.sandipan.das@amd.com/
https://lore.kernel.org/kvm/b075ec13e95167da1f2229626713c91d291bb75b.1784097178.git.sandipan.das@amd.com/
https://lore.kernel.org/all/106bc3a4660653cb7750f3ae07a4dbca42c2762c.1786356229.git.sandipan.das@amd.com/

base-commit: 6cf46b37b0356b3408f8ba949e10462cc71b533d
             (tag: kvm-x86-next-2026.08.07)

Previous versions can be found at:
rfc: https://lore.kernel.org/kvm/cover.1762960531.git.sandipan.das@amd.com/

Changes in v1:
  * Introduce mediated PMU capabilities for hardware PMU virtualization
    features instead of perf PMU capabilities.
  * Make VNMI a hard requirement since AVIC can be inhibited.
  * Introduce new PMU ops to access VMCB PMU state instead of relying
    on host-initiated {get,set}_msr() ops.
  * Make VMCB PMU state accessible via KVM_{GET,SET}_MSRS for migration.
  * Implement support for nested SVM.
  * Disable support for SEV-ES and SEV-SNP guests explicitly.

Sandipan Das (8):
  KVM: x86/pmu: Add mediated PMU capability flags
  KVM: x86/pmu: Add PMU ops for vendor state sync
  KVM: x86/pmu: Add support for hardware-switched PMU
  x86/cpufeatures: Add PerfCtrVirt feature bit
  KVM: SVM: Add VMCB fields for PMC virtualization
  KVM: SVM: Add support for hardware-switched PMU
  KVM: nSVM: Add support for hardware-switched PMU
  KVM: SEV: Disallow the use of hardware-switched PMU

 arch/x86/include/asm/cpufeatures.h     |   1 +
 arch/x86/include/asm/kvm-x86-pmu-ops.h |   3 +
 arch/x86/include/asm/kvm_host.h        |  11 +++
 arch/x86/include/asm/svm.h             |  13 ++-
 arch/x86/kvm/cpuid.c                   |   1 +
 arch/x86/kvm/pmu.c                     | 102 +++++++++++++++++----
 arch/x86/kvm/pmu.h                     |  29 ++++++
 arch/x86/kvm/svm/nested.c              |  45 +++++++++-
 arch/x86/kvm/svm/pmu.c                 | 117 +++++++++++++++++++++++++
 arch/x86/kvm/svm/sev.c                 |   3 +
 arch/x86/kvm/svm/svm.c                 |  58 +++++++++++-
 arch/x86/kvm/svm/svm.h                 |  19 ++++
 arch/x86/kvm/vmx/pmu_intel.c           |   2 +
 arch/x86/kvm/vmx/vmx.c                 |   3 +-
 arch/x86/kvm/x86.c                     |   2 +
 15 files changed, 384 insertions(+), 25 deletions(-)

-- 
2.53.0

[PATCH 1/8] KVM: x86/pmu: Add mediated PMU capability flags

[Sandipan Das]

When mediated PMU is enabled, the guest PMU state is currently saved
and restored in software. On capable processors, this can be offloaded
to hardware. Hence, introduce the notion of VM-scoped mediated PMU
capabilities to help KVM determine the available hardware assists.

When set, the KVM_MEDIATED_PMU_CAP_HW_SWITCHED flag indicates that
hardware is responsible for switching the guest PMU state. When
software-switched, the guest PMU state continues to reside in struct
kvm_pmu. However, when hardware-switched, the guest PMU state resides
in a vendor save area.

While KVM_MEDIATED_PMU_CAP_HW_FILTERED remains unused currently, when
set, it indicates that hardware is also responsible for assisting in
guest event filtering for further reduction of overhead resulting from
intercepting event selector writes.

The capabilities are expected to be set in the vm_init() call. All
capabilities are cleared if kvm_arch_vcpu_precreate() fails to create
mediated PMU.

Signed-off-by: Sandipan Das <sandipan.das@amd.com>
---
 arch/x86/include/asm/kvm_host.h | 11 +++++++++++
 arch/x86/kvm/pmu.h              | 20 ++++++++++++++++++++
 arch/x86/kvm/x86.c              |  2 ++
 3 files changed, 33 insertions(+)

diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 283847619ff8..338df11e2a4d 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -715,6 +715,16 @@ enum kvm_only_cpuid_leafs {
 	NKVMCAPINTS = NR_KVM_CPU_CAPS - NCAPINTS,
 };
 
+/*
+ * Track how the guest PMU state is saved and restored, as this can be done
+ * either by software or, if capable, by hardware. When hardware manages the
+ * guest PMU state, the state resides in a vendor save area.
+ */
+#define KVM_MEDIATED_PMU_CAP_HW_SWITCHED	BIT(0)
+#define KVM_MEDIATED_PMU_CAP_HW_FILTERED	BIT(1)
+#define KVM_MEDIATED_PMU_CAP_VALID		(KVM_MEDIATED_PMU_CAP_HW_SWITCHED	| \
+						 KVM_MEDIATED_PMU_CAP_HW_FILTERED)
+
 struct kvm_vcpu_arch {
 	/*
 	 * rip and regs accesses must go through
@@ -1284,6 +1294,7 @@ struct kvm_arch {
 	bool bus_lock_detection_enabled;
 	bool enable_pmu;
 	bool created_mediated_pmu;
+	u32 mediated_pmu_caps;
 
 	u32 notify_window;
 	u32 notify_vmexit_flags;
diff --git a/arch/x86/kvm/pmu.h b/arch/x86/kvm/pmu.h
index 090c9bbb74f4..1aa46d3e2339 100644
--- a/arch/x86/kvm/pmu.h
+++ b/arch/x86/kvm/pmu.h
@@ -88,6 +88,26 @@ static inline bool kvm_vcpu_has_mediated_pmu(struct kvm_vcpu *vcpu)
 	return enable_mediated_pmu && vcpu_to_pmu(vcpu)->version;
 }
 
+static inline bool kvm_vcpu_has_mediated_pmu_caps(struct kvm_vcpu *vcpu, u32 caps)
+{
+	return kvm_vcpu_has_mediated_pmu(vcpu) &&
+	       !!(vcpu->kvm->arch.mediated_pmu_caps & caps);
+}
+
+#define kvm_set_mediated_pmu_caps(kvm, caps)				\
+do {									\
+	BUILD_BUG_ON(!__builtin_constant_p(caps) ||			\
+		     ((caps) & ~KVM_MEDIATED_PMU_CAP_VALID));		\
+	(kvm)->arch.mediated_pmu_caps |= (caps);			\
+} while (0)
+
+#define kvm_clr_mediated_pmu_caps(kvm, caps)				\
+do {									\
+	BUILD_BUG_ON(!__builtin_constant_p(caps) ||			\
+		     ((caps) & ~KVM_MEDIATED_PMU_CAP_VALID));		\
+	(kvm)->arch.mediated_pmu_caps &= ~(caps);			\
+} while (0)
+
 /*
  * KVM tracks all counters in 64-bit bitmaps, with general purpose counters
  * mapped to bits 31:0 and fixed counters mapped to 63:32, e.g. fixed counter 0
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index d94b59140c45..4287982638b6 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -9326,6 +9326,7 @@ int kvm_arch_vcpu_precreate(struct kvm *kvm, unsigned int id)
 		if (irqchip_in_kernel(kvm)) {
 			r = perf_create_mediated_pmu();
 			if (r) {
+				kvm->arch.mediated_pmu_caps = 0;
 				pr_warn_ratelimited(PERF_MEDIATED_PMU_MSG);
 				return r;
 			}
@@ -9334,6 +9335,7 @@ int kvm_arch_vcpu_precreate(struct kvm *kvm, unsigned int id)
 			kvm->arch.enable_pmu = false;
 		}
 	}
+
 	return 0;
 }
 
-- 
2.53.0

[PATCH 2/8] KVM: x86/pmu: Add PMU ops for vendor state sync

[Sandipan Das]

When hardware-switched mediated PMU is enabled, the guest PMU state
resides in a vendor save area. Because of this, after every VMEXIT,
the state inside struct kvm_pmu can go out of sync with the vendor
save area. Since KVM expects struct kvm_pmu to have the correct guest
PMU state during event filtering and instruction emulation, introduce
new PMU ops to opportunistically synchronize the state of PMU MSRs
between the two backing stores.

The new ops are directional. get_vendor_state() copies data from the
vendor save area to the corresponding member of struct kvm_pmu while
set_vendor_state() does the opposite.

Signed-off-by: Sandipan Das <sandipan.das@amd.com>
---
 arch/x86/include/asm/kvm-x86-pmu-ops.h | 3 +++
 arch/x86/kvm/pmu.c                     | 5 ++++-
 arch/x86/kvm/pmu.h                     | 2 ++
 3 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/kvm-x86-pmu-ops.h b/arch/x86/include/asm/kvm-x86-pmu-ops.h
index 4a223c2793e3..4a617162d4c0 100644
--- a/arch/x86/include/asm/kvm-x86-pmu-ops.h
+++ b/arch/x86/include/asm/kvm-x86-pmu-ops.h
@@ -29,6 +29,9 @@ KVM_X86_PMU_OP_OPTIONAL_RET0(pmc_is_disabled_in_current_mode)
 KVM_X86_PMU_OP_OPTIONAL(write_global_ctrl)
 KVM_X86_PMU_OP(mediated_load)
 KVM_X86_PMU_OP(mediated_put)
+
+KVM_X86_PMU_OP_OPTIONAL_RET0(get_vendor_state)
+KVM_X86_PMU_OP_OPTIONAL_RET0(set_vendor_state)
 #endif
 
 #undef KVM_X86_PMU_OP
diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c
index a7d60c8785cd..5ede3ee968e3 100644
--- a/arch/x86/kvm/pmu.c
+++ b/arch/x86/kvm/pmu.c
@@ -166,8 +166,11 @@ void kvm_init_pmu_capability(struct kvm_pmu_ops *pmu_ops)
 	    !pmu_ops->is_mediated_pmu_supported(&kvm_host_pmu))
 		enable_mediated_pmu = false;
 
-	if (!enable_mediated_pmu)
+	if (!enable_mediated_pmu) {
 		pmu_ops->write_global_ctrl = NULL;
+		pmu_ops->get_vendor_state = NULL;
+		pmu_ops->set_vendor_state = NULL;
+	}
 
 	if (!enable_pmu) {
 		memset(&kvm_pmu_cap, 0, sizeof(kvm_pmu_cap));
diff --git a/arch/x86/kvm/pmu.h b/arch/x86/kvm/pmu.h
index 1aa46d3e2339..5d6f6863ad40 100644
--- a/arch/x86/kvm/pmu.h
+++ b/arch/x86/kvm/pmu.h
@@ -42,6 +42,8 @@ struct kvm_pmu_ops {
 	void (*mediated_load)(struct kvm_vcpu *vcpu);
 	void (*mediated_put)(struct kvm_vcpu *vcpu);
 	void (*write_global_ctrl)(u64 global_ctrl);
+	int (*get_vendor_state)(struct kvm_vcpu *vcpu, u32 msr);
+	int (*set_vendor_state)(struct kvm_vcpu *vcpu, u32 msr);
 
 	const u64 EVENTSEL_EVENT;
 	const int MAX_NR_GP_COUNTERS;
-- 
2.53.0

[PATCH 3/8] KVM: x86/pmu: Add support for hardware-switched PMU

[Sandipan Das]

Implement the hardware-switched mode of mediated PMU. The key design
differences from the software-switched mode are listed below.
  * Hardware handles context-switching of the guest PMU state.
  * The guest PMU state resides in a vendor save area (such as VMCB or
    VMCS) instead of struct kvm_pmu.

When enabled, the load and put functionality of mediated PMU simplifies
to just scheduling the active host events in and out.

Event filtering and instruction emulation require the ability to change
the guest PMU state in software. Since struct kvm_pmu is not guaranteed
to always have the correct state, make use of the new vendor state sync
ops to access MSR states directly from a vendor save area. These are
also used to read the latest counter value when RDPMC is intercepted
for legacy guests with fewer counters.

The software-switched mode continues to be the default for hardware
lacking the features to support this capability.

Signed-off-by: Sandipan Das <sandipan.das@amd.com>
---
 arch/x86/kvm/pmu.c           | 97 +++++++++++++++++++++++++++++-------
 arch/x86/kvm/pmu.h           |  7 +++
 arch/x86/kvm/svm/pmu.c       |  2 +
 arch/x86/kvm/svm/svm.c       |  3 +-
 arch/x86/kvm/vmx/pmu_intel.c |  2 +
 arch/x86/kvm/vmx/vmx.c       |  3 +-
 6 files changed, 95 insertions(+), 19 deletions(-)

diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c
index 5ede3ee968e3..4c918aa6d6f3 100644
--- a/arch/x86/kvm/pmu.c
+++ b/arch/x86/kvm/pmu.c
@@ -130,6 +130,24 @@ void kvm_pmu_ops_update(const struct kvm_pmu_ops *pmu_ops)
 #undef __KVM_X86_PMU_OP
 }
 
+static void kvm_pmu_get_vendor_state(struct kvm_vcpu *vcpu, u32 msr)
+{
+	if (!kvm_vcpu_has_mediated_pmu_caps(vcpu, KVM_MEDIATED_PMU_CAP_HW_SWITCHED))
+		return;
+
+	if (kvm_pmu_call(get_vendor_state)(vcpu, msr))
+		kvm_pmu_warn_vendor_state(msr);
+}
+
+static void kvm_pmu_set_vendor_state(struct kvm_vcpu *vcpu, u32 msr)
+{
+	if (!kvm_vcpu_has_mediated_pmu_caps(vcpu, KVM_MEDIATED_PMU_CAP_HW_SWITCHED))
+		return;
+
+	if (kvm_pmu_call(set_vendor_state)(vcpu, msr))
+		kvm_pmu_warn_vendor_state(msr);
+}
+
 void kvm_init_pmu_capability(struct kvm_pmu_ops *pmu_ops)
 {
 	bool is_intel = boot_cpu_data.x86_vendor == X86_VENDOR_INTEL;
@@ -200,6 +218,27 @@ void kvm_handle_guest_mediated_pmi(void)
 	kvm_make_request(KVM_REQ_PMI, vcpu);
 }
 
+static __always_inline u32 fixed_counter_msr(u32 idx)
+{
+	return kvm_pmu_ops.FIXED_COUNTER_BASE + idx * kvm_pmu_ops.MSR_STRIDE;
+}
+
+static __always_inline u32 gp_counter_msr(u32 idx)
+{
+	return kvm_pmu_ops.GP_COUNTER_BASE + idx * kvm_pmu_ops.MSR_STRIDE;
+}
+
+static __always_inline u32 gp_eventsel_msr(u32 idx)
+{
+	return kvm_pmu_ops.GP_EVENTSEL_BASE + idx * kvm_pmu_ops.MSR_STRIDE;
+}
+
+static __always_inline u32 pmc_counter_msr(struct kvm_pmc *pmc)
+{
+	return pmc_is_gp(pmc) ? gp_counter_msr(pmc->idx) :
+				fixed_counter_msr(pmc->idx - KVM_FIXED_PMC_BASE_IDX);
+}
+
 static inline void __kvm_perf_overflow(struct kvm_pmc *pmc, bool in_pmi)
 {
 	struct kvm_pmu *pmu = pmc_to_pmu(pmc);
@@ -553,18 +592,23 @@ static void kvm_mediated_pmu_refresh_event_filter(struct kvm_pmc *pmc)
 {
 	bool allowed = pmc_is_locally_enabled(pmc) && pmc_is_event_allowed(pmc);
 	struct kvm_pmu *pmu = pmc_to_pmu(pmc);
+	struct kvm_vcpu *vcpu = pmc->vcpu;
 
 	if (pmc_is_gp(pmc)) {
 		pmc->eventsel_hw &= ~ARCH_PERFMON_EVENTSEL_ENABLE;
 		if (allowed)
 			pmc->eventsel_hw |= pmc->eventsel &
 					    ARCH_PERFMON_EVENTSEL_ENABLE;
+
+		kvm_pmu_set_vendor_state(vcpu, gp_eventsel_msr(pmc->idx));
 	} else {
 		u64 mask = intel_fixed_bits_by_idx(pmc->idx - KVM_FIXED_PMC_BASE_IDX, 0xf);
 
 		pmu->fixed_ctr_ctrl_hw &= ~mask;
 		if (allowed)
 			pmu->fixed_ctr_ctrl_hw |= pmu->fixed_ctr_ctrl & mask;
+
+		kvm_pmu_set_vendor_state(vcpu, kvm_pmu_ops.FIXED_COUNTER_CTRL);
 	}
 }
 
@@ -772,6 +816,8 @@ int kvm_pmu_rdpmc(struct kvm_vcpu *vcpu, unsigned idx, u64 *data)
 	    kvm_is_cr0_bit_set(vcpu, X86_CR0_PE))
 		return 1;
 
+	kvm_pmu_get_vendor_state(vcpu, pmc_counter_msr(pmc));
+
 	*data = pmc_read_counter(pmc) & mask;
 	return 0;
 }
@@ -855,10 +901,12 @@ int kvm_pmu_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 	switch (msr) {
 	case MSR_CORE_PERF_GLOBAL_STATUS:
 	case MSR_AMD64_PERF_CNTR_GLOBAL_STATUS:
+		kvm_pmu_get_vendor_state(vcpu, kvm_pmu_ops.PERF_GLOBAL_STATUS);
 		msr_info->data = pmu->global_status;
 		break;
 	case MSR_AMD64_PERF_CNTR_GLOBAL_CTL:
 	case MSR_CORE_PERF_GLOBAL_CTRL:
+		kvm_pmu_get_vendor_state(vcpu, kvm_pmu_ops.PERF_GLOBAL_CTRL);
 		msr_info->data = pmu->global_ctrl;
 		break;
 	case MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_CLR:
@@ -898,6 +946,7 @@ int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 			return 1;
 
 		pmu->global_status = data;
+		kvm_pmu_set_vendor_state(vcpu, kvm_pmu_ops.PERF_GLOBAL_STATUS);
 		break;
 	case MSR_AMD64_PERF_CNTR_GLOBAL_CTL:
 		data &= ~pmu->global_ctrl_rsvd;
@@ -917,6 +966,8 @@ int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 		 */
 		if (kvm_vcpu_has_mediated_pmu(vcpu))
 			kvm_pmu_call(write_global_ctrl)(data);
+
+		kvm_pmu_set_vendor_state(vcpu, kvm_pmu_ops.PERF_GLOBAL_CTRL);
 		break;
 	case MSR_CORE_PERF_GLOBAL_OVF_CTRL:
 		/*
@@ -927,12 +978,18 @@ int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 			return 1;
 		fallthrough;
 	case MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_CLR:
-		if (!msr_info->host_initiated)
+		if (!msr_info->host_initiated) {
+			kvm_pmu_get_vendor_state(vcpu, kvm_pmu_ops.PERF_GLOBAL_STATUS);
 			pmu->global_status &= ~data;
+			kvm_pmu_set_vendor_state(vcpu, kvm_pmu_ops.PERF_GLOBAL_STATUS);
+		}
 		break;
 	case MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_SET:
-		if (!msr_info->host_initiated)
+		if (!msr_info->host_initiated) {
+			kvm_pmu_get_vendor_state(vcpu, kvm_pmu_ops.PERF_GLOBAL_STATUS);
 			pmu->global_status |= data & ~pmu->global_status_rsvd;
+			kvm_pmu_set_vendor_state(vcpu, kvm_pmu_ops.PERF_GLOBAL_STATUS);
+		}
 		break;
 	default:
 		kvm_pmu_mark_pmc_in_use(vcpu, msr_info->index);
@@ -1020,6 +1077,8 @@ void kvm_pmu_refresh(struct kvm_vcpu *vcpu)
 	if (kvm_vcpu_has_mediated_pmu(vcpu))
 		kvm_pmu_call(write_global_ctrl)(pmu->global_ctrl);
 
+	kvm_pmu_set_vendor_state(vcpu, kvm_pmu_ops.PERF_GLOBAL_CTRL);
+
 	bitmap_set(pmu->all_valid_pmc_idx, 0, pmu->nr_arch_gp_counters);
 	bitmap_set(pmu->all_valid_pmc_idx, KVM_FIXED_PMC_BASE_IDX,
 		   pmu->nr_arch_fixed_counters);
@@ -1142,6 +1201,9 @@ static void kvm_pmu_trigger_event(struct kvm_vcpu *vcpu,
 	if (bitmap_empty(event_pmcs, X86_PMC_IDX_MAX))
 		return;
 
+	kvm_pmu_get_vendor_state(vcpu, kvm_pmu_ops.PERF_GLOBAL_CTRL);
+	kvm_pmu_get_vendor_state(vcpu, kvm_pmu_ops.PERF_GLOBAL_STATUS);
+
 	if (!kvm_pmu_has_perf_global_ctrl(pmu))
 		bitmap_copy(bitmap, event_pmcs, X86_PMC_IDX_MAX);
 	else if (!bitmap_and(bitmap, event_pmcs,
@@ -1150,11 +1212,17 @@ static void kvm_pmu_trigger_event(struct kvm_vcpu *vcpu,
 
 	idx = srcu_read_lock(&vcpu->kvm->srcu);
 	kvm_for_each_pmc(pmu, pmc, i, bitmap) {
+		kvm_pmu_get_vendor_state(vcpu, pmc_counter_msr(pmc));
+
 		if (!pmc_is_event_allowed(pmc) || !cpl_is_matched(pmc))
 			continue;
 
 		kvm_pmu_incr_counter(pmc);
+		kvm_pmu_set_vendor_state(vcpu, pmc_counter_msr(pmc));
 	}
+
+	kvm_pmu_set_vendor_state(vcpu, kvm_pmu_ops.PERF_GLOBAL_STATUS);
+
 	srcu_read_unlock(&vcpu->kvm->srcu, idx);
 }
 
@@ -1313,21 +1381,6 @@ int kvm_vm_ioctl_set_pmu_event_filter(struct kvm *kvm, void __user *argp)
 	return r;
 }
 
-static __always_inline u32 fixed_counter_msr(u32 idx)
-{
-	return kvm_pmu_ops.FIXED_COUNTER_BASE + idx * kvm_pmu_ops.MSR_STRIDE;
-}
-
-static __always_inline u32 gp_counter_msr(u32 idx)
-{
-	return kvm_pmu_ops.GP_COUNTER_BASE + idx * kvm_pmu_ops.MSR_STRIDE;
-}
-
-static __always_inline u32 gp_eventsel_msr(u32 idx)
-{
-	return kvm_pmu_ops.GP_EVENTSEL_BASE + idx * kvm_pmu_ops.MSR_STRIDE;
-}
-
 static void kvm_pmu_load_guest_pmcs(struct kvm_vcpu *vcpu)
 {
 	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
@@ -1364,6 +1417,10 @@ void kvm_mediated_pmu_load(struct kvm_vcpu *vcpu)
 
 	perf_load_guest_context();
 
+	/* Guest PMU state is restored by hardware */
+	if (kvm_vcpu_has_mediated_pmu_caps(vcpu, KVM_MEDIATED_PMU_CAP_HW_SWITCHED))
+		return;
+
 	/*
 	 * Explicitly clear PERF_GLOBAL_CTRL, as "loading" the guest's context
 	 * disables all individual counters (if any were enabled), but doesn't
@@ -1423,6 +1480,12 @@ void kvm_mediated_pmu_put(struct kvm_vcpu *vcpu)
 
 	lockdep_assert_irqs_disabled();
 
+	/* Guest PMU state is saved by hardware */
+	if (kvm_vcpu_has_mediated_pmu_caps(vcpu, KVM_MEDIATED_PMU_CAP_HW_SWITCHED)) {
+		perf_put_guest_context();
+		return;
+	}
+
 	/*
 	 * Defer handling of PERF_GLOBAL_CTRL to vendor code.  On Intel, it's
 	 * atomically cleared on VM-Exit, i.e. doesn't need to be clear here.
diff --git a/arch/x86/kvm/pmu.h b/arch/x86/kvm/pmu.h
index 5d6f6863ad40..42d33072ece9 100644
--- a/arch/x86/kvm/pmu.h
+++ b/arch/x86/kvm/pmu.h
@@ -50,8 +50,10 @@ struct kvm_pmu_ops {
 	const int MIN_NR_GP_COUNTERS;
 
 	const u32 PERF_GLOBAL_CTRL;
+	const u32 PERF_GLOBAL_STATUS;
 	const u32 GP_EVENTSEL_BASE;
 	const u32 GP_COUNTER_BASE;
+	const u32 FIXED_COUNTER_CTRL;
 	const u32 FIXED_COUNTER_BASE;
 	const u32 MSR_STRIDE;
 };
@@ -110,6 +112,11 @@ do {									\
 	(kvm)->arch.mediated_pmu_caps &= ~(caps);			\
 } while (0)
 
+static inline void kvm_pmu_warn_vendor_state(u32 msr)
+{
+	WARN_ONCE(1, "accessing unsupported vendor save slot for MSR 0x%x\n", msr);
+}
+
 /*
  * KVM tracks all counters in 64-bit bitmaps, with general purpose counters
  * mapped to bits 31:0 and fixed counters mapped to 63:32, e.g. fixed counter 0
diff --git a/arch/x86/kvm/svm/pmu.c b/arch/x86/kvm/svm/pmu.c
index c18286545a7a..5dccf8776368 100644
--- a/arch/x86/kvm/svm/pmu.c
+++ b/arch/x86/kvm/svm/pmu.c
@@ -321,8 +321,10 @@ struct kvm_pmu_ops amd_pmu_ops __initdata = {
 	.MIN_NR_GP_COUNTERS = AMD64_NUM_COUNTERS,
 
 	.PERF_GLOBAL_CTRL = MSR_AMD64_PERF_CNTR_GLOBAL_CTL,
+	.PERF_GLOBAL_STATUS = MSR_AMD64_PERF_CNTR_GLOBAL_STATUS,
 	.GP_EVENTSEL_BASE = MSR_F15H_PERF_CTL0,
 	.GP_COUNTER_BASE = MSR_F15H_PERF_CTR0,
+	.FIXED_COUNTER_CTRL = 0,
 	.FIXED_COUNTER_BASE = 0,
 	.MSR_STRIDE = 2,
 };
diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
index 9d607b98bd06..2e39b5e752cd 100644
--- a/arch/x86/kvm/svm/svm.c
+++ b/arch/x86/kvm/svm/svm.c
@@ -4618,7 +4618,8 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu, u64 run_flags)
 
 	kvm_clear_available_registers(vcpu, SVM_REGS_LAZY_LOAD_SET);
 
-	if (!msr_write_intercepted(svm, MSR_AMD64_PERF_CNTR_GLOBAL_CTL))
+	if (!kvm_vcpu_has_mediated_pmu_caps(vcpu, KVM_MEDIATED_PMU_CAP_HW_SWITCHED) &&
+	    !msr_write_intercepted(svm, MSR_AMD64_PERF_CNTR_GLOBAL_CTL))
 		rdmsrq(MSR_AMD64_PERF_CNTR_GLOBAL_CTL, vcpu_to_pmu(vcpu)->global_ctrl);
 
 	trace_kvm_exit(vcpu, KVM_ISA_SVM);
diff --git a/arch/x86/kvm/vmx/pmu_intel.c b/arch/x86/kvm/vmx/pmu_intel.c
index bfa8612fb450..d0723814410e 100644
--- a/arch/x86/kvm/vmx/pmu_intel.c
+++ b/arch/x86/kvm/vmx/pmu_intel.c
@@ -853,8 +853,10 @@ struct kvm_pmu_ops intel_pmu_ops __initdata = {
 	.MIN_NR_GP_COUNTERS = 1,
 
 	.PERF_GLOBAL_CTRL = MSR_CORE_PERF_GLOBAL_CTRL,
+	.PERF_GLOBAL_STATUS = MSR_CORE_PERF_GLOBAL_STATUS,
 	.GP_EVENTSEL_BASE = MSR_P6_EVNTSEL0,
 	.GP_COUNTER_BASE = MSR_IA32_PMC0,
+	.FIXED_COUNTER_CTRL = MSR_CORE_PERF_FIXED_CTR_CTRL,
 	.FIXED_COUNTER_BASE = MSR_CORE_PERF_FIXED_CTR0,
 	.MSR_STRIDE = 1,
 };
diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index e3bfe6aca1a0..7d8c032c1897 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -7383,7 +7383,8 @@ static void vmx_refresh_guest_perf_global_control(struct kvm_vcpu *vcpu)
 	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
 	struct vcpu_vmx *vmx = to_vmx(vcpu);
 
-	if (msr_write_intercepted(vmx, MSR_CORE_PERF_GLOBAL_CTRL))
+	if (kvm_vcpu_has_mediated_pmu_caps(vcpu, KVM_MEDIATED_PMU_CAP_HW_SWITCHED) ||
+	    msr_write_intercepted(vmx, MSR_CORE_PERF_GLOBAL_CTRL))
 		return;
 
 	if (!cpu_has_save_perf_global_ctrl()) {
-- 
2.53.0

[PATCH 4/8] x86/cpufeatures: Add PerfCtrVirt feature bit

[Sandipan Das]

Define a feature flag for bit 8 of CPUID leaf 0x8000000A EDX which
indicates support for PMC virtualization. Hypervisors can use this
feature to offload the guest PMU context switch to hardware. Hence,
this is a required host capability for enabling hardware-switched
mediated PMU in KVM.

Signed-off-by: Sandipan Das <sandipan.das@amd.com>
---
 arch/x86/include/asm/cpufeatures.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
index 30bb8fdf2a8b..b551ba381b6e 100644
--- a/arch/x86/include/asm/cpufeatures.h
+++ b/arch/x86/include/asm/cpufeatures.h
@@ -384,6 +384,7 @@
 #define X86_FEATURE_VMCBCLEAN		(15*32+ 5) /* "vmcb_clean" VMCB clean bits support */
 #define X86_FEATURE_FLUSHBYASID		(15*32+ 6) /* "flushbyasid" Flush-by-ASID support */
 #define X86_FEATURE_DECODEASSISTS	(15*32+ 7) /* "decodeassists" Decode Assists support */
+#define X86_FEATURE_PERFCTR_VIRT	(15*32+ 8) /* PMC virtualization support */
 #define X86_FEATURE_PAUSEFILTER		(15*32+10) /* "pausefilter" Filtered pause intercept */
 #define X86_FEATURE_PFTHRESHOLD		(15*32+12) /* "pfthreshold" Pause filter threshold */
 #define X86_FEATURE_AVIC		(15*32+13) /* "avic" Virtual Interrupt Controller */
-- 
2.53.0

[PATCH 5/8] KVM: SVM: Add VMCB fields for PMC virtualization

[Sandipan Das]

When PMC virtualization (X86_FEATURE_PERFCTR_VIRT) is supported and the
feature is enabled, additional save slots are available in the VMCB for
the following MSRs.
  * Performance Counter Global Control (MSR 0xc0000301) (Swap Type C)
  * Performance Counter Global Status (MSR 0xc0000300) (Swap Type A)
  * Performance Event Select (MSR 0xc0010200..0xc001020a) (Swap Type C)
  * Performance Event Counter (MSR 0xc0010201..0xc001020b) (Swap Type C)

Define the additional VMCB fields that will be used by hardware to save
and restore the guest PMU state.

Signed-off-by: Sandipan Das <sandipan.das@amd.com>
---
 arch/x86/include/asm/svm.h | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/arch/x86/include/asm/svm.h b/arch/x86/include/asm/svm.h
index aa63431ba92c..73f17af22d6c 100644
--- a/arch/x86/include/asm/svm.h
+++ b/arch/x86/include/asm/svm.h
@@ -338,7 +338,12 @@ struct vmcb_save_area {
 	u8 cpl;
 	u8 reserved_0xcc[4];
 	u64 efer;
-	u8 reserved_0xd8[112];
+	u8 reserved_0xd8[8];
+	struct {
+		u64 perf_ctl;
+		u64 perf_ctr;
+	} __packed pmc[6];
+	u8 reserved_0x140[8];
 	u64 cr4;
 	u64 cr3;
 	u64 cr0;
@@ -346,7 +351,9 @@ struct vmcb_save_area {
 	u64 dr6;
 	u64 rflags;
 	u64 rip;
-	u8 reserved_0x180[88];
+	u8 reserved_0x180[72];
+	u64 perf_cntr_global_status;
+	u64 perf_cntr_global_control;
 	u64 rsp;
 	u64 s_cet;
 	u64 ssp;
@@ -575,6 +582,7 @@ static inline void __unused_size_checks(void)
 	BUILD_BUG_RESERVED_OFFSET(vmcb_save_area, 0xa0);
 	BUILD_BUG_RESERVED_OFFSET(vmcb_save_area, 0xcc);
 	BUILD_BUG_RESERVED_OFFSET(vmcb_save_area, 0xd8);
+	BUILD_BUG_RESERVED_OFFSET(vmcb_save_area, 0x140);
 	BUILD_BUG_RESERVED_OFFSET(vmcb_save_area, 0x180);
 	BUILD_BUG_RESERVED_OFFSET(vmcb_save_area, 0x248);
 	BUILD_BUG_RESERVED_OFFSET(vmcb_save_area, 0x298);
-- 
2.53.0

[PATCH 6/8] KVM: SVM: Add support for hardware-switched PMU

[Sandipan Das]

Zen 5 processors introduced the ability to offload guest PMU context
switches to hardware using a feature called PMC virtualization. Use the
feature, detected using X86_FEATURE_PERFCTR_VIRT, to implement the new
hardware-switched mode of mediated PMU.

The feature is enabled by setting bit 3 of misc_ctl2 residing in the
VMCB control area. Since the guest PMU state now resides in the VMCB
save area, implement the vendor state sync PMU ops for use during event
filtering, instruction emulation and RDPMC interception to access the
latest PMU state.

The vendor state sync calls always use MSR_F15H_PERF_{CTL,CTR}x indices
because of how GP_EVENTSEL_BASE and GP_COUNTER_BASE are set in the SVM
kvm_pmu_ops. For legacy models, where X86_FEATURE_PERFCTR_CORE is not
set, these are translated to the equivalent MSR_K7_{EVNTSEL,PERFCTR}x
indices, as get_gp_pmc_amd() would otherwise return NULL.

The underlying implementation depends on the availability of either
VNMI or AVIC for guest PMI delivery. Synthesized overflows such as
those resulting from incrementing counters due to instruction emulation
are still injected through the KVM_REQ_PMI processing path. Since AVIC
can be inhibited, make VNMI the hard requirement.

When both VNMI and AVIC are enabled, AVIC gets preference in hardware.
The main advantage of using AVIC is that it lets the guest change the
delivery mode in the LVTPC. Unlike AVIC, VNMI ignores the LVTPC, as the
APIC is emulated, and always presents PMIs as NMIs.

PMC virtualization does not use a VMCB clean bit. Hardware reloads the
guest PMU state from the VMCB save area on every VMRUN. Hence, any
software updates to the VMCB do not require calling vmcb_mark_dirty().

The "vpmc" vendor module parameter is introduced to toggle the feature.
Compared to software-switched mediated PMU, enabling the feature
reduces the world switch overhead. Hence it is enabled by default when
the host supports X86_FEATURE_PERFCTR_VIRT and also has mediated PMU
enabled (kvm_amd.enable_mediated_pmu=1).

Signed-off-by: Sandipan Das <sandipan.das@amd.com>
---
 arch/x86/include/asm/svm.h |   1 +
 arch/x86/kvm/svm/pmu.c     | 115 +++++++++++++++++++++++++++++++++++++
 arch/x86/kvm/svm/svm.c     |  51 ++++++++++++++++
 arch/x86/kvm/svm/svm.h     |   1 +
 4 files changed, 168 insertions(+)

diff --git a/arch/x86/include/asm/svm.h b/arch/x86/include/asm/svm.h
index 73f17af22d6c..e1c8a333e86f 100644
--- a/arch/x86/include/asm/svm.h
+++ b/arch/x86/include/asm/svm.h
@@ -247,6 +247,7 @@ struct __attribute__ ((__packed__)) vmcb_control_area {
 
 #define SVM_MISC2_ENABLE_V_LBR	BIT_ULL(0)
 #define SVM_MISC2_ENABLE_V_VMLOAD_VMSAVE	BIT_ULL(1)
+#define SVM_MISC2_ENABLE_V_PMC	BIT_ULL(3)
 
 #define SVM_TSC_RATIO_RSVD	0xffffff0000000000ULL
 #define SVM_TSC_RATIO_MIN	0x0000000000000001ULL
diff --git a/arch/x86/kvm/svm/pmu.c b/arch/x86/kvm/svm/pmu.c
index 5dccf8776368..7d3a151b6cde 100644
--- a/arch/x86/kvm/svm/pmu.c
+++ b/arch/x86/kvm/svm/pmu.c
@@ -125,6 +125,89 @@ static bool amd_is_valid_msr(struct kvm_vcpu *vcpu, u32 msr)
 	return amd_msr_idx_to_pmc(vcpu, msr);
 }
 
+static u32 amd_pmu_adjust_msr_idx(struct kvm_vcpu *vcpu, u32 msr)
+{
+	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
+
+	if (!guest_cpu_cap_has(vcpu, X86_FEATURE_PERFCTR_CORE) &&
+	    msr >= MSR_F15H_PERF_CTL0 &&
+	    msr <= MSR_F15H_PERF_CTR0 + 2 * pmu->nr_arch_gp_counters)
+		msr = ((msr & 0x1) ? MSR_K7_PERFCTR0 : MSR_K7_EVNTSEL0) +
+		      ((msr - MSR_F15H_PERF_CTL0) / 2);
+
+	return msr;
+}
+
+static int amd_get_vendor_state(struct kvm_vcpu *vcpu, u32 msr)
+{
+	struct vmcb_save_area *save = &to_svm(vcpu)->vmcb->save;
+	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
+	struct kvm_pmc *pmc;
+
+	/* MSR_PERF_CNTR_GLOBAL_* */
+	switch (msr) {
+	case MSR_AMD64_PERF_CNTR_GLOBAL_STATUS:
+		pmu->global_status = save->perf_cntr_global_status;
+		return 0;
+	case MSR_AMD64_PERF_CNTR_GLOBAL_CTL:
+		pmu->global_ctrl = save->perf_cntr_global_control;
+		return 0;
+	}
+
+	msr = amd_pmu_adjust_msr_idx(vcpu, msr);
+
+	/* MSR_PERFCTRn */
+	pmc = get_gp_pmc_amd(pmu, msr, PMU_TYPE_COUNTER);
+	if (pmc) {
+		pmc->counter = save->pmc[pmc->idx].perf_ctr;
+		return 0;
+	}
+
+	/* MSR_EVNTSELn */
+	pmc = get_gp_pmc_amd(pmu, msr, PMU_TYPE_EVNTSEL);
+	if (pmc) {
+		pmc->eventsel_hw = save->pmc[pmc->idx].perf_ctl;
+		return 0;
+	}
+
+	return 1;
+}
+
+static int amd_set_vendor_state(struct kvm_vcpu *vcpu, u32 msr)
+{
+	struct vmcb_save_area *save = &to_svm(vcpu)->vmcb->save;
+	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
+	struct kvm_pmc *pmc;
+
+	/* MSR_PERF_CNTR_GLOBAL_* */
+	switch (msr) {
+	case MSR_AMD64_PERF_CNTR_GLOBAL_STATUS:
+		save->perf_cntr_global_status = pmu->global_status;
+		return 0;
+	case MSR_AMD64_PERF_CNTR_GLOBAL_CTL:
+		save->perf_cntr_global_control = pmu->global_ctrl;
+		return 0;
+	}
+
+	msr = amd_pmu_adjust_msr_idx(vcpu, msr);
+
+	/* MSR_PERFCTRn */
+	pmc = get_gp_pmc_amd(pmu, msr, PMU_TYPE_COUNTER);
+	if (pmc) {
+		save->pmc[pmc->idx].perf_ctr = pmc->counter & pmc_bitmask(pmc);
+		return 0;
+	}
+
+	/* MSR_EVNTSELn */
+	pmc = get_gp_pmc_amd(pmu, msr, PMU_TYPE_EVNTSEL);
+	if (pmc) {
+		save->pmc[pmc->idx].perf_ctl = pmc->eventsel_hw;
+		return 0;
+	}
+
+	return 1;
+}
+
 static int amd_pmu_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 {
 	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
@@ -134,6 +217,10 @@ static int amd_pmu_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 	/* MSR_PERFCTRn */
 	pmc = get_gp_pmc_amd(pmu, msr, PMU_TYPE_COUNTER);
 	if (pmc) {
+		if (msr_info->host_initiated &&
+		    kvm_vcpu_has_mediated_pmu_caps(vcpu, KVM_MEDIATED_PMU_CAP_HW_SWITCHED) &&
+		    amd_get_vendor_state(vcpu, msr))
+			kvm_pmu_warn_vendor_state(msr);
 		msr_info->data = pmc_read_counter(pmc);
 		return 0;
 	}
@@ -158,6 +245,10 @@ static int amd_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 	pmc = get_gp_pmc_amd(pmu, msr, PMU_TYPE_COUNTER);
 	if (pmc) {
 		pmc_write_counter(pmc, data);
+		if (msr_info->host_initiated &&
+		    kvm_vcpu_has_mediated_pmu_caps(vcpu, KVM_MEDIATED_PMU_CAP_HW_SWITCHED) &&
+		    amd_set_vendor_state(vcpu, msr))
+			kvm_pmu_warn_vendor_state(msr);
 		return 0;
 	}
 	/* MSR_EVNTSELn */
@@ -174,6 +265,9 @@ static int amd_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 			else
 				__clear_bit(pmc->idx, pmu->pmc_has_mode_specific_enables);
 
+			if (kvm_vcpu_has_mediated_pmu_caps(vcpu, KVM_MEDIATED_PMU_CAP_HW_SWITCHED))
+				pmc->eventsel_hw = data;
+
 			kvm_pmu_request_counter_reprogram(pmc);
 		}
 		return 0;
@@ -239,6 +333,24 @@ static void amd_pmu_init(struct kvm_vcpu *vcpu)
 	}
 }
 
+static void amd_pmu_reset(struct kvm_vcpu *vcpu)
+{
+	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
+	struct vmcb_save_area *save = &to_svm(vcpu)->vmcb->save;
+	int i;
+
+	if (!kvm_vcpu_has_mediated_pmu_caps(vcpu, KVM_MEDIATED_PMU_CAP_HW_SWITCHED))
+		return;
+
+	for (i = 0; i < pmu->nr_arch_gp_counters; i++) {
+		save->pmc[i].perf_ctl = 0;
+		save->pmc[i].perf_ctr = 0;
+	}
+
+	save->perf_cntr_global_control = 0;
+	save->perf_cntr_global_status = 0;
+}
+
 static bool amd_pmu_is_mediated_pmu_supported(struct x86_pmu_capability *host_pmu)
 {
 	return host_pmu->version >= 2;
@@ -311,10 +423,13 @@ struct kvm_pmu_ops amd_pmu_ops __initdata = {
 	.refresh = amd_pmu_refresh,
 	.init = amd_pmu_init,
 	.pmc_is_disabled_in_current_mode = amd_pmc_is_disabled_in_current_mode,
+	.reset = amd_pmu_reset,
 
 	.is_mediated_pmu_supported = amd_pmu_is_mediated_pmu_supported,
 	.mediated_load = amd_mediated_pmu_load,
 	.mediated_put = amd_mediated_pmu_put,
+	.get_vendor_state = amd_get_vendor_state,
+	.set_vendor_state = amd_set_vendor_state,
 
 	.EVENTSEL_EVENT = AMD64_EVENTSEL_EVENT,
 	.MAX_NR_GP_COUNTERS = KVM_MAX_NR_AMD_GP_COUNTERS,
diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
index 2e39b5e752cd..079b1a56e995 100644
--- a/arch/x86/kvm/svm/svm.c
+++ b/arch/x86/kvm/svm/svm.c
@@ -179,6 +179,9 @@ module_param(vnmi, bool, 0444);
 
 module_param(enable_mediated_pmu, bool, 0444);
 
+bool vpmc = true;
+module_param(vpmc, bool, 0444);
+
 static bool __ro_after_init svm_gp_erratum_intercept = true;
 
 static u8 rsm_ins_bytes[] = "\x0f\xaa";
@@ -1267,6 +1270,9 @@ static void init_vmcb(struct kvm_vcpu *vcpu, bool init_event)
 
 	svm_hv_init_vmcb(vmcb);
 
+	if (kvm_vcpu_has_mediated_pmu_caps(vcpu, KVM_MEDIATED_PMU_CAP_HW_SWITCHED))
+		control->misc_ctl2 |= SVM_MISC2_ENABLE_V_PMC;
+
 	kvm_make_request(KVM_REQ_RECALC_INTERCEPTS, vcpu);
 
 	vmcb_mark_all_dirty(vmcb);
@@ -3563,6 +3569,30 @@ static void dump_vmcb(struct kvm_vcpu *vcpu)
 	       "excp_from:", save->last_excp_from,
 	       "excp_to:", save->last_excp_to);
 
+	if (kvm_vcpu_has_mediated_pmu_caps(vcpu, KVM_MEDIATED_PMU_CAP_HW_SWITCHED)) {
+		pr_err("%-15s %016llx %-13s %016llx\n",
+		       "perf_ctl0:", save->pmc[0].perf_ctl,
+		       "perf_ctr0:", save->pmc[0].perf_ctr);
+		pr_err("%-15s %016llx %-13s %016llx\n",
+		       "perf_ctl1:", save->pmc[1].perf_ctl,
+		       "perf_ctr1:", save->pmc[1].perf_ctr);
+		pr_err("%-15s %016llx %-13s %016llx\n",
+		       "perf_ctl2:", save->pmc[2].perf_ctl,
+		       "perf_ctr2:", save->pmc[2].perf_ctr);
+		pr_err("%-15s %016llx %-13s %016llx\n",
+		       "perf_ctl3:", save->pmc[3].perf_ctl,
+		       "perf_ctr3:", save->pmc[3].perf_ctr);
+		pr_err("%-15s %016llx %-13s %016llx\n",
+		       "perf_ctl4:", save->pmc[4].perf_ctl,
+		       "perf_ctr4:", save->pmc[4].perf_ctr);
+		pr_err("%-15s %016llx %-13s %016llx\n",
+		       "perf_ctl5:", save->pmc[5].perf_ctl,
+		       "perf_ctr5:", save->pmc[5].perf_ctr);
+		pr_err("%-15s %016llx %-13s %016llx\n",
+		       "perf_cntr_global_control:", save->perf_cntr_global_control,
+		       "perf_cntr_global_status:", save->perf_cntr_global_status);
+	}
+
 	if (is_sev_es_guest(vcpu)) {
 		struct sev_es_save_area *vmsa = (struct sev_es_save_area *)save;
 
@@ -4452,6 +4482,14 @@ static noinstr void svm_vcpu_enter_exit(struct kvm_vcpu *vcpu, unsigned enter_fl
 
 	amd_clear_divider();
 
+	/*
+	 * On #VMEXIT, PerfCntrGlobalCtl goes back to its reset state since
+	 * its save slot is of Swap Type C. All enable bits are set but PMC
+	 * virtualization requires them to be cleared before VMRUN.
+	 */
+	if (kvm_vcpu_has_mediated_pmu_caps(vcpu, KVM_MEDIATED_PMU_CAP_HW_SWITCHED))
+		wrmsrq(MSR_AMD64_PERF_CNTR_GLOBAL_CTL, 0);
+
 	if (is_sev_es_guest(vcpu))
 		__svm_sev_es_vcpu_run(svm, enter_flags,
 				      sev_es_host_save_area(sd));
@@ -5289,6 +5327,9 @@ static void svm_vm_destroy(struct kvm *kvm)
 
 static int svm_vm_init(struct kvm *kvm)
 {
+	if (vpmc)
+		kvm_set_mediated_pmu_caps(kvm, KVM_MEDIATED_PMU_CAP_HW_SWITCHED);
+
 	sev_vm_init(kvm);
 
 	if (!pause_filter_count || !pause_filter_thresh)
@@ -5735,6 +5776,16 @@ static __init int svm_hardware_setup(void)
 	if (!enable_pmu)
 		pr_info("PMU virtualization is disabled\n");
 
+	/*
+	 * PMC virtualization does not raise host PMIs that need to be injected.
+	 * Instead, it requires VNMI or AVIC for guest PMI delivery. AVIC can,
+	 * however, get inhibited so make VNMI the hard requirement.
+	 */
+	vpmc = vpmc && vnmi && enable_mediated_pmu &&
+	       cpu_feature_enabled(X86_FEATURE_PERFCTR_VIRT);
+	if (vpmc)
+		pr_info("PMC virtualization supported\n");
+
 	svm_set_cpu_caps();
 
 	kvm_caps.inapplicable_quirks &= ~KVM_X86_QUIRK_CD_NW_CLEARED;
diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h
index 66b44b54608e..eee26f6ce10c 100644
--- a/arch/x86/kvm/svm/svm.h
+++ b/arch/x86/kvm/svm/svm.h
@@ -53,6 +53,7 @@ extern int vgif;
 extern bool intercept_smi;
 extern bool vnmi;
 extern int lbrv;
+extern bool vpmc;
 
 extern int tsc_aux_uret_slot __ro_after_init;
 
-- 
2.53.0

[PATCH 7/8] KVM: nSVM: Add support for hardware-switched PMU

[Sandipan Das]

Add nested support for PMC virtualization and advertise
X86_FEATURE_PERFCTR_VIRT so that a L1 hypervisor can use it for its L2
guests. As long as the L0 hypervisor uses hardware-switched mediated
PMU for L1, it is used while L1 runs L2 guests as well, irrespective of
what L1 does.

Since hardware runs L2 from vmcb02, not vmcb12, enable PMC
virtualization in vmcb02 whenever it is enabled in vmcb01, so that
hardware switches the PMU state using the vmcb02 save area. The state
to switch depends on whether L1 enables PMC virtualization for L2.

An L1 hypervisor enables PMC virtualization for L2 by setting bit 3 of
misc_ctl2 in vmcb12. Following commit 84dc9fd0354d ("KVM: nSVM: Cache
all used fields from VMCB12"), cache the PMU-related fields to avoid
TOCTOU hazards. Copy the cached L2 PMU state into vmcb02 for L2 VMRUNs,
and back into vmcb12 on nested VMEXITs, so that L1 observes the latest
L2 PMU state.

When L1 does not enable PMC virtualization for L2, the PMU state to
switch is the one in the vmcb01 save area. Copy it into vmcb02 for L2
VMRUNs, and back into vmcb01 on nested VMEXITs, so that the latest PMU
state is preserved for L1.

The counter reprogramming done on nested transitions writes into the
vmcb02 save area, so run it after copying the PMU state into vmcb02;
otherwise the copy overwrites it.

Signed-off-by: Sandipan Das <sandipan.das@amd.com>
---
 arch/x86/kvm/cpuid.c      |  1 +
 arch/x86/kvm/svm/nested.c | 45 ++++++++++++++++++++++++++++++++++++---
 arch/x86/kvm/svm/svm.c    |  4 ++++
 arch/x86/kvm/svm/svm.h    | 18 ++++++++++++++++
 4 files changed, 65 insertions(+), 3 deletions(-)

diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c
index ddb022cb203a..ce4f984bc6e1 100644
--- a/arch/x86/kvm/cpuid.c
+++ b/arch/x86/kvm/cpuid.c
@@ -1217,6 +1217,7 @@ void kvm_initialize_cpu_caps(void)
 		VENDOR_F(VGIF),
 		VENDOR_F(VNMI),
 		VENDOR_F(SVME_ADDR_CHK),
+		VENDOR_F(PERFCTR_VIRT),
 	);
 
 	kvm_cpu_cap_init(CPUID_8000_001F_EAX,
diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c
index 73f37b050d0a..a9f9caeb7840 100644
--- a/arch/x86/kvm/svm/nested.c
+++ b/arch/x86/kvm/svm/nested.c
@@ -605,6 +605,7 @@ static void __nested_copy_vmcb_save_to_cache(struct vmcb_save_area_cached *to,
 	to->g_pat = from->g_pat;
 
 	svm_copy_lbrs(to, from);
+	svm_copy_pmcs(to, from);
 }
 
 void nested_copy_vmcb_save_to_cache(struct vcpu_svm *svm,
@@ -741,6 +742,17 @@ static bool nested_vmcb12_has_lbrv(struct kvm_vcpu *vcpu)
 		(to_svm(vcpu)->nested.ctl.misc_ctl2 & SVM_MISC2_ENABLE_V_LBR);
 }
 
+static bool nested_vmcb12_has_vpmc(struct kvm_vcpu *vcpu)
+{
+	/*
+	 * Since nested AVIC is not supported, L2 PMIs can only be delivered
+	 * via VNMI, so make it a hard requirement.
+	 */
+	return guest_cpu_cap_has(vcpu, X86_FEATURE_PERFCTR_VIRT) &&
+		nested_vnmi_enabled(to_svm(vcpu)) &&
+		(to_svm(vcpu)->nested.ctl.misc_ctl2 & SVM_MISC2_ENABLE_V_PMC);
+}
+
 static void nested_vmcb02_prepare_save(struct vcpu_svm *svm)
 {
 	struct vmcb_ctrl_area_cached *control = &svm->nested.ctl;
@@ -823,6 +835,11 @@ static void nested_vmcb02_prepare_save(struct vcpu_svm *svm)
 	}
 	vmcb_mark_dirty(vmcb02, VMCB_LBR);
 	svm_update_lbrv(&svm->vcpu);
+
+	if (nested_vmcb12_has_vpmc(vcpu))
+		svm_copy_pmcs(&vmcb02->save, save);
+	else if (kvm_vcpu_has_mediated_pmu_caps(vcpu, KVM_MEDIATED_PMU_CAP_HW_SWITCHED))
+		svm_copy_pmcs(&vmcb02->save, &vmcb01->save);
 }
 
 static inline bool is_evtinj_soft(u32 evtinj)
@@ -863,7 +880,6 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm)
 
 	/* Enter Guest-Mode */
 	enter_guest_mode(vcpu);
-	svm_pmu_handle_nested_transition(svm);
 
 	/*
 	 * Filled at exit: exit_code, exit_info_1, exit_info_2, exit_int_info,
@@ -986,6 +1002,10 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm)
 
 	/* SVM_MISC2_ENABLE_V_LBR is controlled by svm_update_lbrv() */
 
+	/* L0 uses hardware-switched mode for both vmcb01 and vmcb02 */
+	if (kvm_vcpu_has_mediated_pmu_caps(vcpu, KVM_MEDIATED_PMU_CAP_HW_SWITCHED))
+		vmcb02->control.misc_ctl2 |= SVM_MISC2_ENABLE_V_PMC;
+
 	if (!nested_vmcb_needs_vls_intercept(svm))
 		vmcb02->control.misc_ctl2 |= SVM_MISC2_ENABLE_V_VMLOAD_VMSAVE;
 
@@ -1065,6 +1085,9 @@ int enter_svm_guest_mode(struct kvm_vcpu *vcpu, u64 vmcb12_gpa, bool from_vmrun)
 	nested_vmcb02_prepare_control(svm);
 	nested_vmcb02_prepare_save(svm);
 
+	if (!nested_vmcb12_has_vpmc(vcpu))
+		svm_pmu_handle_nested_transition(svm);
+
 	ret = nested_svm_load_cr3(&svm->vcpu, svm->nested.save.cr3,
 				  nested_npt_enabled(svm), from_vmrun);
 	if (ret)
@@ -1228,6 +1251,9 @@ void svm_copy_vmrun_state(struct vmcb_save_area *to_save,
 		svm_copy_lbrs(to_save, from_save);
 		to_save->dbgctl &= ~DEBUGCTL_RESERVED_BITS;
 	}
+
+	if (kvm_cpu_cap_has(X86_FEATURE_PERFCTR_VIRT))
+		svm_copy_pmcs(to_save, from_save);
 }
 
 void svm_copy_vmloadsave_state(struct vmcb *to_vmcb, struct vmcb *from_vmcb)
@@ -1300,6 +1326,9 @@ static int nested_svm_vmexit_update_vmcb12(struct kvm_vcpu *vcpu)
 	if (nested_vmcb12_has_lbrv(vcpu))
 		svm_copy_lbrs(&vmcb12->save, &vmcb02->save);
 
+	if (nested_vmcb12_has_vpmc(vcpu))
+		svm_copy_pmcs(&vmcb12->save, &vmcb02->save);
+
 	vmcb12->control.event_inj	  = 0;
 	vmcb12->control.event_inj_err	  = 0;
 	vmcb12->control.int_ctl           = svm->nested.ctl.int_ctl;
@@ -1325,7 +1354,9 @@ void nested_svm_vmexit(struct vcpu_svm *svm)
 
 	/* Exit Guest-Mode */
 	leave_guest_mode(vcpu);
-	svm_pmu_handle_nested_transition(svm);
+
+	if (!nested_vmcb12_has_vpmc(vcpu))
+		svm_pmu_handle_nested_transition(svm);
 
 	svm->nested.vmcb12_gpa = 0;
 
@@ -1381,6 +1412,10 @@ void nested_svm_vmexit(struct vcpu_svm *svm)
 
 	svm_update_lbrv(vcpu);
 
+	if (!nested_vmcb12_has_vpmc(vcpu) &&
+	    kvm_vcpu_has_mediated_pmu_caps(vcpu, KVM_MEDIATED_PMU_CAP_HW_SWITCHED))
+		svm_copy_pmcs(&vmcb01->save, &vmcb02->save);
+
 	if (vnmi) {
 		if (vmcb02->control.int_ctl & V_NMI_BLOCKING_MASK)
 			vmcb01->control.int_ctl |= V_NMI_BLOCKING_MASK;
@@ -1545,7 +1580,8 @@ void svm_leave_nested(struct kvm_vcpu *vcpu)
 		 * into PMU state from arbitrary contexts (e.g. to avoid using
 		 * stale state).
 		 */
-		__svm_pmu_handle_nested_transition(svm, true);
+		if (!nested_vmcb12_has_vpmc(vcpu))
+			__svm_pmu_handle_nested_transition(svm, true);
 
 		svm_switch_vmcb(svm, &svm->vmcb01);
 
@@ -2074,6 +2110,9 @@ static int svm_set_nested_state(struct kvm_vcpu *vcpu,
 
 	nested_vmcb02_prepare_control(svm);
 
+	if (!nested_vmcb12_has_vpmc(vcpu))
+		svm_pmu_handle_nested_transition(svm);
+
 	/*
 	 * Any previously restored state (e.g. KVM_SET_SREGS) would mark fields
 	 * dirty in vmcb01 instead of vmcb02, so mark all of vmcb02 dirty here.
diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
index 079b1a56e995..2023d38bba63 100644
--- a/arch/x86/kvm/svm/svm.c
+++ b/arch/x86/kvm/svm/svm.c
@@ -5583,6 +5583,10 @@ static __init void svm_set_cpu_caps(void)
 		if (vnmi)
 			kvm_cpu_cap_set(X86_FEATURE_VNMI);
 
+		/* Nested AVIC is not supported, so VNMI must be enabled */
+		if (vpmc && vnmi)
+			kvm_cpu_cap_set(X86_FEATURE_PERFCTR_VIRT);
+
 		/* Nested VM can receive #VMEXIT instead of triggering #GP */
 		kvm_cpu_cap_set(X86_FEATURE_SVME_ADDR_CHK);
 	}
diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h
index eee26f6ce10c..3db6b73e8163 100644
--- a/arch/x86/kvm/svm/svm.h
+++ b/arch/x86/kvm/svm/svm.h
@@ -175,6 +175,12 @@ struct vmcb_save_area_cached {
 	u64 br_to;
 	u64 last_excp_from;
 	u64 last_excp_to;
+	struct {
+		u64 perf_ctl;
+		u64 perf_ctr;
+	} pmc[6];
+	u64 perf_cntr_global_status;
+	u64 perf_cntr_global_control;
 };
 
 struct vmcb_ctrl_area_cached {
@@ -806,6 +812,18 @@ do {								\
 	(to)->last_excp_to	= (from)->last_excp_to;		\
 } while (0)
 
+#define svm_copy_pmcs(to, from)							\
+do {										\
+	int i;									\
+										\
+	(to)->perf_cntr_global_control = (from)->perf_cntr_global_control;	\
+	(to)->perf_cntr_global_status  = (from)->perf_cntr_global_status;	\
+	for (i = 0; i < kvm_pmu_cap.num_counters_gp; i++) {			\
+		(to)->pmc[i].perf_ctl = (from)->pmc[i].perf_ctl;		\
+		(to)->pmc[i].perf_ctr = (from)->pmc[i].perf_ctr;		\
+	}									\
+} while (0)
+
 void svm_vcpu_free_msrpm(void *msrpm);
 void svm_enable_lbrv(struct kvm_vcpu *vcpu);
 void svm_update_lbrv(struct kvm_vcpu *vcpu);
-- 
2.53.0

[PATCH 8/8] KVM: SEV: Disallow the use of hardware-switched PMU

[Sandipan Das]

For hardware-switched mediated PMU, the guest PMU state resides in the
VMCB save area instead of struct kvm_pmu. To inspect or modify that
state in software, KVM uses the vendor state sync PMU ops.

When SEV-ES or SEV-SNP is used, these ops cannot work as the guest PMU
state resides in an encrypted VMSA that KVM can neither read nor write.
Clear KVM_MEDIATED_PMU_CAP_HW_SWITCHED for such guests so that they
fall back to the software-switched mode.

Signed-off-by: Sandipan Das <sandipan.das@amd.com>
---
 arch/x86/kvm/svm/sev.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index fcb41dfde4c0..8e8cd4feb16a 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -534,6 +534,9 @@ static int __sev_guest_init(struct kvm *kvm, struct kvm_sev_cmd *argp,
 	if (snp_active)
 		sev->vmsa_features |= SVM_SEV_FEAT_SNP_ACTIVE;
 
+	if (es_active)
+		kvm_clr_mediated_pmu_caps(kvm, KVM_MEDIATED_PMU_CAP_HW_SWITCHED);
+
 	ret = sev_asid_new(sev, vm_type);
 	if (ret)
 		goto e_no_asid;
-- 
2.53.0

Re: [PATCH 4/8] x86/cpufeatures: Add PerfCtrVirt feature bit

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] The patch modifies `arch/x86/include/asm/cpufeatures.h` but fails to update its synchronized copy in `tools/arch/x86/include/asm/cpufeatures.h`.
--

--- Patch [4]: [PATCH 4/8] x86/cpufeatures: Add PerfCtrVirt feature bit ---
commit d0af0746d5778097e8c3fa7f7763e5d0b909732c
Author: Sandipan Das <sandipan.das@amd.com>

x86/cpufeatures: Add PerfCtrVirt feature bit

This commit defines a feature flag for bit 8 of CPUID leaf 0x8000000A EDX
which indicates support for PMC virtualization. Hypervisors can use this
feature to offload the guest PMU context switch to hardware. This enables
hardware-switched mediated PMU in KVM.

> diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
> index 1b4a48bff18f7..d6281870efe02 100644
> --- a/arch/x86/include/asm/cpufeatures.h
> +++ b/arch/x86/include/asm/cpufeatures.h
> @@ -374,6 +374,7 @@
>  #define X86_FEATURE_VMCBCLEAN		(15*32+ 5) /* "vmcb_clean" VMCB clean bits support */
>  #define X86_FEATURE_FLUSHBYASID		(15*32+ 6) /* "flushbyasid" Flush-by-ASID support */
>  #define X86_FEATURE_DECODEASSISTS	(15*32+ 7) /* "decodeassists" Decode Assists support */
> +#define X86_FEATURE_PERFCTR_VIRT	(15*32+ 8) /* PMC virtualization support */

[Severity: Low]
Should this addition also be mirrored in the tools directory?

When arch/x86/include/asm/cpufeatures.h is modified, it usually requires a
synchronized update in tools/arch/x86/include/asm/cpufeatures.h.

Does omitting this update trigger a kernel ABI header differences warning
from check-headers.sh during the tools/perf build?

>  #define X86_FEATURE_PAUSEFILTER		(15*32+10) /* "pausefilter" Filtered pause intercept */
>  #define X86_FEATURE_PFTHRESHOLD		(15*32+12) /* "pfthreshold" Pause filter threshold */
>  #define X86_FEATURE_AVIC		(15*32+13) /* "avic" Virtual Interrupt Controller */

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1786345201.git.sandipan.das@amd.com?part=4

Re: [PATCH 2/8] KVM: x86/pmu: Add PMU ops for vendor state sync

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] The newly introduced PMU ops (get_vendor_state, set_vendor_state) are never invoked by KVM nor implemented by any vendor PMU, rendering them dead code.
--

--- Patch [2]: [PATCH 2/8] KVM: x86/pmu: Add PMU ops for vendor state sync ---
commit 2b7ba0292c6d4a2eb66eaca07b256dee5f7af75a
Author: Sandipan Das <sandipan.das@amd.com>

KVM: x86/pmu: Add PMU ops for vendor state sync

This commit introduces new PMU operations get_vendor_state and set_vendor_state
to KVM's PMU architecture to allow synchronizing guest PMU state with a vendor
save area. It updates the capabilities initialization to nullify these operations
if the mediated PMU is not enabled.

> diff --git a/arch/x86/kvm/pmu.h b/arch/x86/kvm/pmu.h
> index f60856c48462b..2b1a8d6433734 100644
> --- a/arch/x86/kvm/pmu.h
> +++ b/arch/x86/kvm/pmu.h
> @@ -42,6 +42,8 @@ struct kvm_pmu_ops {
>  	void (*mediated_load)(struct kvm_vcpu *vcpu);
>  	void (*mediated_put)(struct kvm_vcpu *vcpu);
>  	void (*write_global_ctrl)(u64 global_ctrl);
> +	int (*get_vendor_state)(struct kvm_vcpu *vcpu, u32 msr);
> +	int (*set_vendor_state)(struct kvm_vcpu *vcpu, u32 msr);

[Severity: Low]
Are these new PMU operations actually invoked anywhere in KVM or implemented by
any vendor PMU in this patch? It appears they might be dead code, as there are
no callers or vendor implementations (e.g., for AMD or Intel) provided, despite
the commit message mentioning opportunistically synchronizing the state.

>  
>  	const u64 EVENTSEL_EVENT;
>  	const int MAX_NR_GP_COUNTERS;

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1786345201.git.sandipan.das@amd.com?part=2

Re: [PATCH 1/8] KVM: x86/pmu: Add mediated PMU capability flags

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] Transient failure in `perf_create_mediated_pmu` permanently clears `mediated_pmu_caps`, corrupting VM capability state on retry.
- [Medium] `kvm_vcpu_has_mediated_pmu_caps` evaluates to true if *any* capability matches, rather than *all*.
--

--- Patch [1]: [PATCH 1/8] KVM: x86/pmu: Add mediated PMU capability flags ---
commit b775f5357bd23699130ac68abbdd7cd26849e026
Author: Sandipan Das <sandipan.das@amd.com>

KVM: x86/pmu: Add mediated PMU capability flags

This commit introduces VM-scoped mediated PMU capability flags to help KVM
determine available hardware assists for saving and restoring guest PMU
state. It defines flags for hardware-switched and hardware-filtered PMU
states, and manages these capabilities during VM initialization.

> diff --git a/arch/x86/kvm/pmu.h b/arch/x86/kvm/pmu.h
> index a5821d7c87f93..f60856c48462b 100644
> --- a/arch/x86/kvm/pmu.h
> +++ b/arch/x86/kvm/pmu.h
[ ... ]
> @@ -88,6 +88,26 @@ static inline bool kvm_vcpu_has_mediated_pmu(struct kvm_vcpu *vcpu)
>  	return enable_mediated_pmu && vcpu_to_pmu(vcpu)->version;
>  }
>  
> +static inline bool kvm_vcpu_has_mediated_pmu_caps(struct kvm_vcpu *vcpu, u32 caps)
> +{
> +	return kvm_vcpu_has_mediated_pmu(vcpu) &&
> +	       !!(vcpu->kvm->arch.mediated_pmu_caps & caps);

[Severity: Medium]
When a caller checks for multiple capabilities simultaneously, will this
return true if only one of the bits matches, rather than all requested bits?

If a future caller checks kvm_vcpu_has_mediated_pmu_caps() with multiple
flags, it appears it would return true even if only a subset of those flags
are present in mediated_pmu_caps.

[ ... ]
> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index afcac1042947a..9f0e072ff466d 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
[ ... ]
> @@ -12759,6 +12759,7 @@ int kvm_arch_vcpu_precreate(struct kvm *kvm, unsigned int id)
>  		if (irqchip_in_kernel(kvm)) {
>  			r = perf_create_mediated_pmu();
>  			if (r) {
> +				kvm->arch.mediated_pmu_caps = 0;

[Severity: High]
If perf_create_mediated_pmu() returns a transient error like -EBUSY, is it
safe to permanently clear the VM-wide mediated_pmu_caps here?

If userspace receives the error, resolves the host perf conflict, and
retries the KVM_CREATE_VCPU ioctl, the retry might succeed but the VM's
capabilities will have been irreversibly wiped. Does this lead to a silent
downgrade to software switching for the VM on a successful retry?

>  				pr_warn_ratelimited(PERF_MEDIATED_PMU_MSG);
>  				return r;
>  			}

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1786345201.git.sandipan.das@amd.com?part=1

Re: [PATCH 3/8] KVM: x86/pmu: Add support for hardware-switched PMU

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Hardware-switched PMU state (PMCs, event selectors) is not synchronized with the vendor save area during MSR access, VM reset, and event emulation, causing data loss across live migration and incorrect event filtering.
--

--- Patch [3]: [PATCH 3/8] KVM: x86/pmu: Add support for hardware-switched PMU ---
commit 388311da6f37e672745e8b6b16a96e9812fb8ddd
Author: Sandipan Das <sandipan.das@amd.com>

KVM: x86/pmu: Add support for hardware-switched PMU

This patch introduces the hardware-switched mode for the mediated PMU.
It transitions the guest PMU state to reside in a vendor save area
instead of the software kvm_pmu struct. The load and put functionality
is simplified to schedule host events in and out.

> diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c
> --- a/arch/x86/kvm/pmu.c
> +++ b/arch/x86/kvm/pmu.c
> @@ -844,10 +890,12 @@ int kvm_pmu_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
>  	switch (msr) {
>  	case MSR_CORE_PERF_GLOBAL_STATUS:
>  	case MSR_AMD64_PERF_CNTR_GLOBAL_STATUS:
> +		kvm_pmu_get_vendor_state(vcpu, kvm_pmu_ops.PERF_GLOBAL_STATUS);
>  		msr_info->data = pmu->global_status;
>  		break;
>  	case MSR_AMD64_PERF_CNTR_GLOBAL_CTL:
>  	case MSR_CORE_PERF_GLOBAL_CTRL:
> +		kvm_pmu_get_vendor_state(vcpu, kvm_pmu_ops.PERF_GLOBAL_CTRL);
>  		msr_info->data = pmu->global_ctrl;
>  		break;

[Severity: High]
Does this miss synchronizing the PMCs and Event Selectors in the default
switch cases for kvm_pmu_get_msr() and kvm_pmu_set_msr()?

When userspace reads or writes the guest PMU state via KVM_GET_MSRS and
KVM_SET_MSRS, the default block falls through to vendor code:

	default:
		return kvm_pmu_call(get_msr)(vcpu, msr_info);

Without calling kvm_pmu_get_vendor_state() or kvm_pmu_set_vendor_state() for
these MSRs, KVM might return stale software values or fail to commit restored
values to the hardware save area. Since kvm_mediated_pmu_load() skips loading
software state into hardware for the hardware-switched mode, could this cause
data loss across live migration?

[ ... ]

> @@ -1140,11 +1202,17 @@ static void kvm_pmu_trigger_event(struct kvm_vcpu *vcpu,
>  
>  	idx = srcu_read_lock(&vcpu->kvm->srcu);
>  	kvm_for_each_pmc(pmu, pmc, i, bitmap) {
> +		kvm_pmu_get_vendor_state(vcpu, pmc_counter_msr(pmc));
> +
>  		if (!pmc_is_event_allowed(pmc) || !cpl_is_matched(pmc))
>  			continue;
>  
>  		kvm_pmu_incr_counter(pmc);
> +		kvm_pmu_set_vendor_state(vcpu, pmc_counter_msr(pmc));
>  	}

[Severity: High]
Are we evaluating the event filter with stale event selector state here?

While we fetch the latest counter values via kvm_pmu_get_vendor_state(),
pmc_is_event_allowed() and cpl_is_matched() still rely on pmc->eventsel
and pmu->fixed_ctr_ctrl. Without syncing the event selector or fixed counter
control from the vendor save area, might these checks use outdated software
state?

Additionally, how does KVM handle VM resets and instruction emulation tracking
with hardware-switched PMUs?

For resets, kvm_pmu_reset() zeroes out the PMCs and event selectors in the
software structures:

	pmc_stop_counter(pmc);
	pmc->counter = 0;
	pmc->emulated_counter = 0;

	if (pmc_is_gp(pmc)) {
		pmc->eventsel = 0;
		pmc->eventsel_hw = 0;

It doesn't seem to push this reset state down to the vendor save area.

For emulation, MSR writes bypass KVM interception in the hardware-switched
mode. Since KVM_REQ_PMU is never triggered, it appears that
kvm_pmu_recalc_pmc_emulation() won't run, leaving pmc_counting_instructions
and pmc_counting_branches empty. Could this cause KVM to silently fail to
increment counters during instruction and branch emulation?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1786345201.git.sandipan.das@amd.com?part=3

Re: [PATCH 8/8] KVM: SEV: Disallow the use of hardware-switched PMU

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] The patch correctly disables hardware-switched PMU for newly initialized SEV-ES guests to fix a bug where KVM attempts to interact with PMU state in the unencrypted VMCB save area instead of the guest's encrypted VMSA. However, this fix fails to apply to SEV-ES VMs created via migration or mirroring.
--

--- Patch [8]: [PATCH 8/8] KVM: SEV: Disallow the use of hardware-switched PMU ---
commit 0e4fd883c1122d7a244f0bb8754669e617c9a34a
Author: Sandipan Das <sandipan.das@amd.com>

KVM: SEV: Disallow the use of hardware-switched PMU

For hardware-switched mediated PMU, the guest PMU state resides in the
VMCB save area. When SEV-ES or SEV-SNP is used, KVM cannot access the
encrypted VMSA, so this patch clears KVM_MEDIATED_PMU_CAP_HW_SWITCHED for
such guests to force software-switched mode.

> diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
> index 944aaea6501f2..c5df6da8d030f 100644
> --- a/arch/x86/kvm/svm/sev.c
> +++ b/arch/x86/kvm/svm/sev.c
> @@ -534,6 +534,9 @@ static int __sev_guest_init(struct kvm *kvm, struct kvm_sev_cmd *argp,
>  	if (snp_active)
>  		sev->vmsa_features |= SVM_SEV_FEAT_SNP_ACTIVE;
>  
> +	if (es_active)
> +		kvm_clr_mediated_pmu_caps(kvm, KVM_MEDIATED_PMU_CAP_HW_SWITCHED);
> +

[Severity: High]
Does this clearing also need to happen when an SEV-ES VM is created via
migration or mirroring?

Looking at sev_migrate_from(), it copies the es_active state to the
destination VM but does not appear to clear the hardware-switched PMU
capability:

arch/x86/kvm/svm/sev.c:sev_migrate_from() {
	...
	dst->es_active = src->es_active;
	dst->vmsa_features = src->vmsa_features;
	...
}

Similarly, in sev_vm_copy_enc_context_from(), the mirror_sev->es_active flag
is set to true without clearing the PMU capability:

arch/x86/kvm/svm/sev.c:sev_vm_copy_enc_context_from() {
	...
	mirror_sev->es_active = source_sev->es_active;
	...
}

Since KVM unconditionally sets KVM_MEDIATED_PMU_CAP_HW_SWITCHED during VM
creation, could failing to clear it on the destination VMs leave the hardware
PMU feature incorrectly enabled?

If a user migrates or mirrors an SEV-ES VM, might KVM attempt to read/write
PMU state from the unencrypted VMCB save area instead of the guest's
encrypted VMSA, reintroducing the issue this patch aims to fix?

>  	ret = sev_asid_new(sev, vm_type);
>  	if (ret)
>  		goto e_no_asid;

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1786345201.git.sandipan.das@amd.com?part=8

Re: [PATCH 7/8] KVM: nSVM: Add support for hardware-switched PMU

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [Critical] L1 hypervisor can bypass KVM's PMU event filter for L2 guests.
- [High] KVM's software PMU state is not context-switched between L1 and L2, allowing L2 MSR writes to permanently corrupt L1's PMU state and leak state.
- [Medium] KVM silently breaks PMU functionality for L1 and L2 if L1 enables `V_PMC` but L0 does not use hardware-switched mediated PMU.
--

--- Patch [7]: [PATCH 7/8] KVM: nSVM: Add support for hardware-switched PMU ---
commit 93d914b77cd285073e93f03a803ccbca4c41f03f
Author: Sandipan Das <sandipan.das@amd.com>

KVM: nSVM: Add support for hardware-switched PMU

This commit adds nested support for PMC virtualization. It advertises
X86_FEATURE_PERFCTR_VIRT to allow an L1 hypervisor to use it for L2 guests,
switching PMU state using vmcb02's save area.

> diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c
> index 3e6c671a8dc26..666f8ca38a928 100644
> --- a/arch/x86/kvm/svm/nested.c
> +++ b/arch/x86/kvm/svm/nested.c
[ ... ]
> @@ -741,6 +742,17 @@ static bool nested_vmcb12_has_lbrv(struct kvm_vcpu *vcpu)
>  		(to_svm(vcpu)->nested.ctl.misc_ctl2 & SVM_MISC2_ENABLE_V_LBR);
>  }
>  
> +static bool nested_vmcb12_has_vpmc(struct kvm_vcpu *vcpu)
> +{
> +	/*
> +	 * Since nested AVIC is not supported, L2 PMIs can only be delivered
> +	 * via VNMI, so make it a hard requirement.
> +	 */
> +	return guest_cpu_cap_has(vcpu, X86_FEATURE_PERFCTR_VIRT) &&
> +		nested_vnmi_enabled(to_svm(vcpu)) &&
> +		(to_svm(vcpu)->nested.ctl.misc_ctl2 & SVM_MISC2_ENABLE_V_PMC);
> +}

[Severity: Medium]
Will this break PMU functionality if L0 does not use hardware-switched
mediated PMU?

If nested_vmcb12_has_vpmc() returns true because the guest CPUID supports
PERFCTR_VIRT and V_PMC is set in vmcb12, KVM skips the software PMU context
switch in enter_svm_guest_mode() and nested_svm_vmexit().

However, if KVM uses the legacy software PMU
(KVM_MEDIATED_PMU_CAP_HW_SWITCHED is not set), SVM_MISC2_ENABLE_V_PMC is
never set in vmcb02->control.misc_ctl2. Does this result in neither hardware
nor software PMU context switching occurring for both L1 and L2?

[ ... ]
> @@ -1065,6 +1085,9 @@ int enter_svm_guest_mode(struct kvm_vcpu *vcpu, u64 vmcb12_gpa, bool from_vmrun)
>  	nested_vmcb02_prepare_control(svm);
>  	nested_vmcb02_prepare_save(svm);
>  
> +	if (!nested_vmcb12_has_vpmc(vcpu))
> +		svm_pmu_handle_nested_transition(svm);
> +

[Severity: Critical]
By skipping svm_pmu_handle_nested_transition() here, are we omitting the PMU
event filter validation for L2?

The PMU configuration is copied directly from L1's cache into vmcb02 earlier
in nested_vmcb02_prepare_save(). Without calling the transition handler,
kvm_mediated_pmu_refresh_event_filter() is bypassed. Could this allow L1 to
count restricted host events by placing forbidden selectors into vmcb12,
since hardware will load these unvalidated perf_ctl values on VMRUN?

[ ... ]
> @@ -1329,7 +1358,9 @@ void nested_svm_vmexit(struct vcpu_svm *svm)
>  
>  	/* Exit Guest-Mode */
>  	leave_guest_mode(vcpu);
> -	svm_pmu_handle_nested_transition(svm);
> +
> +	if (!nested_vmcb12_has_vpmc(vcpu))
> +		svm_pmu_handle_nested_transition(svm);
>  
>  	svm->nested.vmcb12_gpa = 0;

[Severity: High]
Since svm_pmu_handle_nested_transition() is skipped here, does KVM fail to
restore L1's software PMU state on nested exit?

Because EVNTSEL MSRs are deliberately excluded from MSR merging, L2 writes
to EVNTSEL MSRs will cause an L0 intercept. This intercept updates KVM's
shared software PMU state. If we do not context switch this software state
back to L1's values upon exiting L2, could L2 MSR writes permanently corrupt
L1's PMU tracking state and leak information?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1786345201.git.sandipan.das@amd.com?part=7

Re: [PATCH 6/8] KVM: SVM: Add support for hardware-switched PMU

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider:
- [High] Emulated guest reads and writes to PMU MSRs return stale data or are lost because the vendor state sync is skipped for guest-initiated accesses.
- [High] PMC virtualization is never enabled for the BSP (Boot Strap Processor), leaving its PMU completely broken.
- [High] Nested virtualization (L2) PMU is broken because SVM_MISC2_ENABLE_V_PMC is never propagated to vmcb02.
- [High] Calling `wrmsrq()` inside the `noinstr` function `svm_vcpu_enter_exit()` invokes instrumentable tracepoint code while RCU is not watching.
- [High] The `noinstr` function `svm_vcpu_enter_exit()` calls non-`__always_inline` PMU capability helpers, risking instrumentation in the VMRUN path.
--

--- Patch [6]: [PATCH 6/8] KVM: SVM: Add support for hardware-switched PMU ---
commit d7b31e3949e70dae0c322517e2056c7007227bb4
Author: Sandipan Das <sandipan.das@amd.com>
Subject: KVM: SVM: Add support for hardware-switched PMU

This patch enables PMC virtualization on Zen 5 processors to offload
guest PMU context switches to hardware. It introduces vendor state sync
operations and a module parameter to toggle the feature, integrating it
with the VMCB save area.

> diff --git a/arch/x86/kvm/svm/pmu.c b/arch/x86/kvm/svm/pmu.c
> index 5dccf87763686..7d3a151b6cdee 100644
> --- a/arch/x86/kvm/svm/pmu.c
> +++ b/arch/x86/kvm/svm/pmu.c

[ ... ]

> @@ -134,6 +217,10 @@ static int amd_pmu_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
>  	/* MSR_PERFCTRn */
>  	pmc = get_gp_pmc_amd(pmu, msr, PMU_TYPE_COUNTER);
>  	if (pmc) {
> +		if (msr_info->host_initiated &&
> +		    kvm_vcpu_has_mediated_pmu_caps(vcpu, KVM_MEDIATED_PMU_CAP_HW_SWITCHED) &&
> +		    amd_get_vendor_state(vcpu, msr))
> +			kvm_pmu_warn_vendor_state(msr);
>  		msr_info->data = pmc_read_counter(pmc);
>  		return 0;
>  	}

[Severity: High]
Does gating the vendor state sync behind msr_info->host_initiated break
guest-initiated emulated reads?

When KVM emulates an instruction (e.g., forced emulation or MMIO) that
accesses a PMU counter, host_initiated is false. For reads, this skips the
vendor state sync and returns the stale value in pmc->counter.

> @@ -158,6 +245,10 @@ static int amd_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
>  	pmc = get_gp_pmc_amd(pmu, msr, PMU_TYPE_COUNTER);
>  	if (pmc) {
>  		pmc_write_counter(pmc, data);
> +		if (msr_info->host_initiated &&
> +		    kvm_vcpu_has_mediated_pmu_caps(vcpu, KVM_MEDIATED_PMU_CAP_HW_SWITCHED) &&
> +		    amd_set_vendor_state(vcpu, msr))
> +			kvm_pmu_warn_vendor_state(msr);
>  		return 0;
>  	}

[Severity: High]
Does gating the vendor state sync behind msr_info->host_initiated break
guest-initiated emulated writes?

For emulated writes, KVM will update pmc->counter but skip updating the VMCB
save area because host_initiated is false. Since pmc_write_counter() does not
trigger a PMU refresh for mediated PMUs, the VMCB save area is never updated.
Hardware natively reloads the stale PMU state from the VMCB on the next VMRUN,
silently discarding the guest's emulated write.

> diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
> index ac3d898d21bda..2fa5d85d3c40e 100644
> --- a/arch/x86/kvm/svm/svm.c
> +++ b/arch/x86/kvm/svm/svm.c

[ ... ]

> @@ -1271,6 +1274,9 @@ static void init_vmcb(struct kvm_vcpu *vcpu, bool init_event)
>  
>  	svm_hv_init_vmcb(vmcb);
>  
> +	if (kvm_vcpu_has_mediated_pmu_caps(vcpu, KVM_MEDIATED_PMU_CAP_HW_SWITCHED))
> +		control->misc_ctl2 |= SVM_MISC2_ENABLE_V_PMC;
> +
>  	kvm_make_request(KVM_REQ_RECALC_INTERCEPTS, vcpu);

[Severity: High]
Will PMC virtualization remain disabled for the Boot Strap Processor (BSP)?

During early VCPU creation (svm_vcpu_reset), init_vmcb() is called before
userspace sets the CPUID via KVM_SET_CPUID2. At this point, the PMU version
is 0, so kvm_vcpu_has_mediated_pmu_caps() returns false and
SVM_MISC2_ENABLE_V_PMC is not set.

Unlike APs (which receive an INIT signal that triggers a subsequent
init_vmcb() after CPUID is set), the BSP does not undergo another reset.
During runtime, KVM skips manual PMU context switching because the
capability evaluates to true, but hardware also skips it due to the missing
V_PMC bit in the BSP's VMCB, effectively breaking the PMU for the BSP.

[Severity: High]
Is nested virtualization (L2) PMU broken because SVM_MISC2_ENABLE_V_PMC is
never propagated to vmcb02?

When KVM prepares vmcb02 to run an L2 guest in
nested_vmcb02_prepare_control(), it initializes misc_ctl2 but completely omits
setting SVM_MISC2_ENABLE_V_PMC. KVM will skip manual loading of the PMU state
because kvm_vcpu_has_mediated_pmu_caps() returns true when L2 is running, but
hardware will also skip it because vmcb02 lacks the V_PMC enable bit.

[ ... ]

> @@ -4456,6 +4486,14 @@ static noinstr void svm_vcpu_enter_exit(struct kvm_vcpu *vcpu, unsigned enter_fl
>  
>  	amd_clear_divider();
>  
> +	/*
> +	 * On #VMEXIT, PerfCntrGlobalCtl goes back to its reset state since
> +	 * its save slot is of Swap Type C. All enable bits are set but PMC
> +	 * virtualization requires them to be cleared before VMRUN.
> +	 */
> +	if (kvm_vcpu_has_mediated_pmu_caps(vcpu, KVM_MEDIATED_PMU_CAP_HW_SWITCHED))
> +		wrmsrq(MSR_AMD64_PERF_CNTR_GLOBAL_CTL, 0);
> +
>  	if (is_sev_es_guest(vcpu))
>  		__svm_sev_es_vcpu_run(svm, enter_flags,

[Severity: High]
Does calling kvm_vcpu_has_mediated_pmu_caps() from a noinstr function risk
instrumentation in the VMRUN path?

Functions executed in a noinstr context must be explicitly marked noinstr or
__always_inline. kvm_vcpu_has_mediated_pmu_caps() and its dependency
kvm_vcpu_has_mediated_pmu() are only declared as static inline. If the
compiler compiles them out-of-line, the noinstr protections are stripped,
which can lead to objtool warnings and potential kernel panics if tracing
or instrumentation triggers during this VMRUN entry path.

[Severity: High]
Can calling wrmsrq() inside the noinstr function svm_vcpu_enter_exit() invoke
instrumentable tracepoint code while RCU is not watching?

The wrmsrq() macro wraps native_write_msr(), which conditionally calls
do_trace_write_msr() if the write_msr tracepoint is enabled. Tracepoints
utilize RCU to safely iterate their probe lists. Executing this inside a
noinstr section where RCU is not watching causes lockdep/RCU splats
(RCU used illegally from extended quiescent state) and potential system
panics when tracing is active.

Should this use a non-instrumented wrapper such as native_wrmsrl() or
__wrmsr()?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1786345201.git.sandipan.das@amd.com?part=6