* [RFC PATCH 1/2] KVM: x86/pmu: Add CAP to disable SW accounting of emulated instructions
2026-07-20 19:22 [RFC PATCH 0/2] KVM: x86/pmu: Let userspace disable SW accounting of emulated instructions Luka Absandze
@ 2026-07-20 19:22 ` Luka Absandze
2026-07-20 22:27 ` Sean Christopherson
2026-07-20 19:22 ` [RFC PATCH 2/2] KVM: Documentation: Document KVM_CAP_X86_DISABLE_PMU_SW_ACCOUNTING Luka Absandze
1 sibling, 1 reply; 4+ messages in thread
From: Luka Absandze @ 2026-07-20 19:22 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, kvm
Cc: linux-kernel, linux-doc, Alexander Graf, David Woodhouse,
Luka Absandze
On a host with an emulated vPMU (guest PMU MSR accesses trap and each
guest counter is backed by a host perf_event), the per-emulated-
instruction PMU accounting added by commit 9cd803d496e7 ("KVM: x86: Update
vPMCs when retiring instructions") is catastrophically expensive.
For every instruction KVM emulates, kvm_skip_emulated_instruction(),
the emulator writeback path, and nested VMLAUNCH/VMRESUME call into
the retired-instruction accounting for PERF_COUNT_HW_INSTRUCTIONS and
PERF_COUNT_HW_BRANCH_INSTRUCTIONS. If the guest has a counter
programmed for retired instructions (0xc0) or retired branches (0xc2)
-- e.g. any guest running a profiler with an instructions-retired
event -- this walks the counters, software-increments the matching
one, and requests a full reprogram (KVM_REQ_PMU). The reprogram is
drained on the vCPU's next VM-entry, where reprogram_counter() runs
the full pmc_pause_counter() + perf_event_period() + perf_event_
enable() sequence: ctx->mutex, a ctx_resched() of the PMU context,
and a burst of serialized PMU-MSR writes.
Because the batched reprogram is serviced on the next VM-entry
regardless of which exit preceded it, ordinary exits -- notably the
guest's 1kHz timer tick -- absorb the cost, inflating timer
interrupts into the hundreds of microseconds. With certain workloads
this can escalate to a CSD lockup in the guest.
Add KVM_CAP_X86_DISABLE_PMU_SW_ACCOUNTING, a VM-scoped capability that
takes a bitmask of PERF_COUNT_HW_* event IDs and makes KVM skip the
software accounting of KVM-emulated instructions for those events.
KVM_CHECK_EXTENSION returns the set of events that can be disabled
(PERF_COUNT_HW_INSTRUCTIONS and PERF_COUNT_HW_BRANCH_INSTRUCTIONS --
the only events the retired-instruction path ever triggers).
The only functional change for an opted-in VM is reduced accuracy: a
guest counting instructions-retired or branches-retired undercounts by
the instructions KVM emulates in host context, i.e. the behavior that
predates the accounting cited above. Hardware-executed guest
instructions continue to be counted by the backing perf_event, and its
overflow/PMI path is unchanged. Default behavior (mask 0) is unchanged.
Assisted-by: Claude:claude-opus-4.8
Signed-off-by: Luka Absandze <absandze@amazon.de>
---
arch/x86/include/asm/kvm_host.h | 7 +++++++
arch/x86/kvm/pmu.c | 21 +++++++++++++++++++++
arch/x86/kvm/pmu.h | 9 +++++++++
arch/x86/kvm/x86.c | 15 +++++++++++++++
include/uapi/linux/kvm.h | 1 +
5 files changed, 53 insertions(+)
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 5f6c1ce9673b..522f66c44e6a 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -1560,6 +1560,13 @@ struct kvm_arch {
bool enable_pmu;
bool created_mediated_pmu;
+ /*
+ * Bitmask of PERF_COUNT_HW_* event IDs for which software accounting
+ * of KVM-emulated instructions is disabled (KVM_CAP_X86_DISABLE_PMU_
+ * SW_ACCOUNTING). See kvm_pmu_trigger_event().
+ */
+ u64 pmu_disable_sw_accounting;
+
u32 notify_window;
u32 notify_vmexit_flags;
/*
diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c
index dd1c57593f48..fb3f2df068dc 100644
--- a/arch/x86/kvm/pmu.c
+++ b/arch/x86/kvm/pmu.c
@@ -1145,14 +1145,35 @@ static void kvm_pmu_trigger_event(struct kvm_vcpu *vcpu,
srcu_read_unlock(&vcpu->kvm->srcu, idx);
}
+/*
+ * Whether software accounting of KVM-emulated instructions for @perf_hw_id is
+ * disabled via KVM_CAP_X86_DISABLE_PMU_SW_ACCOUNTING. The capability only
+ * targets the emulated (perf-based) vPMU, where the accounting drives an
+ * expensive counter reprogram; a mediated vPMU increments pmc->counter
+ * directly (and may raise a PMI on overflow), so it is never skipped.
+ */
+static bool kvm_pmu_skip_sw_accounting(struct kvm_vcpu *vcpu, u64 perf_hw_id)
+{
+ if (kvm_vcpu_has_mediated_pmu(vcpu))
+ return false;
+
+ return vcpu->kvm->arch.pmu_disable_sw_accounting & BIT_ULL(perf_hw_id);
+}
+
void kvm_pmu_instruction_retired(struct kvm_vcpu *vcpu)
{
+ if (kvm_pmu_skip_sw_accounting(vcpu, PERF_COUNT_HW_INSTRUCTIONS))
+ return;
+
kvm_pmu_trigger_event(vcpu, vcpu_to_pmu(vcpu)->pmc_counting_instructions);
}
EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_pmu_instruction_retired);
void kvm_pmu_branch_retired(struct kvm_vcpu *vcpu)
{
+ if (kvm_pmu_skip_sw_accounting(vcpu, PERF_COUNT_HW_BRANCH_INSTRUCTIONS))
+ return;
+
kvm_pmu_trigger_event(vcpu, vcpu_to_pmu(vcpu)->pmc_counting_branches);
}
EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_pmu_branch_retired);
diff --git a/arch/x86/kvm/pmu.h b/arch/x86/kvm/pmu.h
index a5821d7c87f9..e34b9e814355 100644
--- a/arch/x86/kvm/pmu.h
+++ b/arch/x86/kvm/pmu.h
@@ -23,6 +23,15 @@
#define KVM_FIXED_PMC_BASE_IDX INTEL_PMC_IDX_FIXED
+/*
+ * Events for which KVM_CAP_X86_DISABLE_PMU_SW_ACCOUNTING can disable software
+ * accounting of emulated instructions. These are the only events KVM ever
+ * passes to kvm_pmu_trigger_event() (from the retired-instruction path).
+ */
+#define KVM_PMU_SW_ACCOUNTING_VALID_MASK \
+ (BIT_ULL(PERF_COUNT_HW_INSTRUCTIONS) | \
+ BIT_ULL(PERF_COUNT_HW_BRANCH_INSTRUCTIONS))
+
struct kvm_pmu_ops {
struct kvm_pmc *(*rdpmc_ecx_to_pmc)(struct kvm_vcpu *vcpu,
unsigned int idx, u64 *mask);
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index afcac1042947..cc8a36f28aea 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -4964,6 +4964,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
case KVM_CAP_DISABLE_QUIRKS2:
r = kvm_caps.supported_quirks;
break;
+ case KVM_CAP_X86_DISABLE_PMU_SW_ACCOUNTING:
+ r = KVM_PMU_SW_ACCOUNTING_VALID_MASK;
+ break;
case KVM_CAP_X86_NOTIFY_VMEXIT:
r = kvm_caps.has_notify_vmexit;
break;
@@ -6730,6 +6733,18 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,
return -EINVAL;
switch (cap->cap) {
+ case KVM_CAP_X86_DISABLE_PMU_SW_ACCOUNTING:
+ r = -EINVAL;
+ if (cap->args[0] & ~KVM_PMU_SW_ACCOUNTING_VALID_MASK)
+ break;
+
+ mutex_lock(&kvm->lock);
+ if (!kvm->created_vcpus) {
+ kvm->arch.pmu_disable_sw_accounting = cap->args[0];
+ r = 0;
+ }
+ mutex_unlock(&kvm->lock);
+ break;
case KVM_CAP_DISABLE_QUIRKS2:
r = -EINVAL;
if (cap->args[0] & ~kvm_caps.supported_quirks)
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 419011097fa8..186054245b80 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -997,6 +997,7 @@ struct kvm_enable_cap {
#define KVM_CAP_S390_KEYOP 247
#define KVM_CAP_S390_VSIE_ESAMODE 248
#define KVM_CAP_S390_HPAGE_2G 249
+#define KVM_CAP_X86_DISABLE_PMU_SW_ACCOUNTING 250
struct kvm_irq_routing_irqchip {
__u32 irqchip;
--
2.47.3
^ permalink raw reply related [flat|nested] 4+ messages in thread* [RFC PATCH 2/2] KVM: Documentation: Document KVM_CAP_X86_DISABLE_PMU_SW_ACCOUNTING
2026-07-20 19:22 [RFC PATCH 0/2] KVM: x86/pmu: Let userspace disable SW accounting of emulated instructions Luka Absandze
2026-07-20 19:22 ` [RFC PATCH 1/2] KVM: x86/pmu: Add CAP to " Luka Absandze
@ 2026-07-20 19:22 ` Luka Absandze
1 sibling, 0 replies; 4+ messages in thread
From: Luka Absandze @ 2026-07-20 19:22 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, kvm
Cc: linux-kernel, linux-doc, Alexander Graf, David Woodhouse,
Luka Absandze
Describe the new VM-scoped capability: its parameter (a bitmask of
PERF_COUNT_HW_* event IDs), the value returned by KVM_CHECK_EXTENSION
(the set of events that can be disabled), the accuracy trade-off, and
why an emulated-vPMU host wants it.
Assisted-by: Claude:claude-opus-4.8
Signed-off-by: Luka Absandze <absandze@amazon.de>
---
Documentation/virt/kvm/api.rst | 32 ++++++++++++++++++++++++++++++++
1 file changed, 32 insertions(+)
diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index a5f9ee92f43e..231a5ba7567c 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -8949,6 +8949,38 @@ enabled, cmma can't be enabled anymore and pfmfi and the storage key
interpretation are disabled. If cmma has already been enabled or the
hpage_2g module parameter is not set to 1, -EINVAL is returned.
+7.48 KVM_CAP_X86_DISABLE_PMU_SW_ACCOUNTING
+------------------------------------------
+
+:Architectures: x86
+:Type: vm
+:Parameters: args[0] - bitmask of PERF_COUNT_HW_* event IDs
+:Returns: 0 on success; -EINVAL if args[0] contains an unsupported event or
+ if the VM already has vCPUs.
+
+By default, KVM software-accounts instructions it emulates against a guest
+PMU counter programmed for retired instructions (PERF_COUNT_HW_INSTRUCTIONS)
+or retired branches (PERF_COUNT_HW_BRANCH_INSTRUCTIONS), by walking the vPMU
+counters and requesting a counter reprogram on the next VM-entry. On hosts
+with an emulated vPMU this reprogram is expensive and, under some workloads,
+inflates timer-interrupt handling enough to trigger guest CSD lockups.
+
+This capability lets userspace disable that software accounting for the given
+events. Setting a bit for an event ID makes KVM skip the accounting for that
+event: instructions KVM emulates in host context go uncounted. Hardware-
+executed guest instructions are still counted by the backing perf_event, and
+its overflow/PMI path is unaffected.
+
+The capability only affects the emulated (perf-based) vPMU. A mediated vPMU
+does not incur the reprogram cost and increments the guest counter directly,
+so accounting is never skipped for it regardless of this setting.
+
+This can only be invoked on a VM prior to the creation of vCPUs; attempting to
+set it once any vCPU exists returns -EINVAL.
+
+KVM_CHECK_EXTENSION returns the bitmask of event IDs that can be disabled.
+args[0] must be a subset of that mask.
+
8. Other capabilities.
======================
--
2.47.3
^ permalink raw reply related [flat|nested] 4+ messages in thread