[RFC PATCH 0/2] KVM: x86/pmu: Let userspace disable SW accounting of emulated instructions

[RFC PATCH 0/2] KVM: x86/pmu: Let userspace disable SW accounting of emulated instructions

[Luka Absandze]

On hosts that run guests with an *emulated* vPMU -- guest PMU MSR
accesses trap into KVM and each guest counter is backed by a host
perf_event -- the per-emulated-instruction PMU accounting added by commit
9cd803d496e7 ("KVM: x86: Update vPMCs when retiring instructions") is
pathologically expensive.

Whenever a guest programs a counter for retired instructions (0xc0) or
retired branches (0xc2) -- routine for any profiler-style workload --
every instruction KVM emulates triggers a software increment plus a
batched KVM_REQ_PMU reprogram. The reprogram is drained on the next
VM-entry regardless of which exit preceded it, so ordinary exits (most
importantly the guest's 1kHz timer tick) absorb a full
reprogram_counter() cycle: ctx->mutex, ctx_resched(), and a burst of
serialized PMU-MSR writes. That inflates timer-interrupt handling into
the hundreds of microseconds and, under load, can escalate to a guest
CSD lockup.

This series adds KVM_CAP_X86_DISABLE_PMU_SW_ACCOUNTING, a VM-scoped
capability that takes a bitmask of PERF_COUNT_HW_* event IDs and makes
KVM skip the software accounting of emulated instructions for those
events. It is opt-in and defaults off; the only cost to an opted-in
guest is that instructions-retired / branches-retired counters
undercount by the instructions KVM emulates in host context, i.e. the
behavior that predates that accounting. Hardware-executed guest
instructions are still counted by the backing perf_event and its
overflow/PMI path is untouched.

  Patch 1 adds the capability and the accounting short-circuit.
  Patch 2 documents it in Documentation/virt/kvm/api.rst.

Measured benefit
================

Host: AMD EPYC 7R13 "Milan", KVM, v7.2-rc4 + this series. Guest: QEMU,
-cpu host,pmu=on, 8 vCPUs. The guest workload keeps an instructions-
retired counter resident on every vCPU and periodically reprograms the
PMU, driving the accounting storm above, while the vCPUs are kept busy
so the 1kHz tick fires. The guest's local-APIC timer interrupt handler
(__sysvec_apic_timer_interrupt) is traced with ftrace function_graph
for 30s (~160k samples per run). The capability is toggled per-VM via
KVM_ENABLE_CAP; nothing else changes between runs.

  Timer-interrupt handler duration:

    metric   accounting ON (baseline)   accounting OFF (CAP)   speedup
    ------   ------------------------   --------------------   -------
    mean            209.8 us                   3.16 us           66x
    p50             256.4 us                   3.14 us           82x
    p99             463.3 us                   4.45 us          104x
    p99.9           915.2 us                   5.03 us          182x
    max             929.7 us                  22.55 us           41x

Some workloads genuinely need the extra precision that the emulated-
instruction accounting provides, and for them the default behavior is
the right trade-off. But for many others the accuracy gained is not
worth the cost it imposes on every exit, and this capability lets those
users opt out of paying it.

Luka Absandze (2):
  KVM: x86/pmu: Add CAP to disable SW accounting of emulated
    instructions
  KVM: Documentation: Document KVM_CAP_X86_DISABLE_PMU_SW_ACCOUNTING

 Documentation/virt/kvm/api.rst  | 24 ++++++++++++++++++++++++
 arch/x86/include/asm/kvm_host.h |  7 +++++++
 arch/x86/kvm/pmu.c              |  6 ++++++
 arch/x86/kvm/pmu.h              |  9 +++++++++
 arch/x86/kvm/x86.c              | 10 ++++++++++
 include/uapi/linux/kvm.h        |  1 +
 6 files changed, 57 insertions(+)

-- 
2.47.3

[RFC PATCH 1/2] KVM: x86/pmu: Add CAP to disable SW accounting of emulated instructions

[Luka Absandze]

On a host with an emulated vPMU (guest PMU MSR accesses trap and each
guest counter is backed by a host perf_event), the per-emulated-
instruction PMU accounting added by commit 9cd803d496e7 ("KVM: x86: Update
vPMCs when retiring instructions") is catastrophically expensive.

For every instruction KVM emulates, kvm_skip_emulated_instruction(),
the emulator writeback path, and nested VMLAUNCH/VMRESUME call into
the retired-instruction accounting for PERF_COUNT_HW_INSTRUCTIONS and
PERF_COUNT_HW_BRANCH_INSTRUCTIONS. If the guest has a counter
programmed for retired instructions (0xc0) or retired branches (0xc2)
-- e.g. any guest running a profiler with an instructions-retired
event -- this walks the counters, software-increments the matching
one, and requests a full reprogram (KVM_REQ_PMU). The reprogram is
drained on the vCPU's next VM-entry, where reprogram_counter() runs
the full pmc_pause_counter() + perf_event_period() + perf_event_
enable() sequence: ctx->mutex, a ctx_resched() of the PMU context,
and a burst of serialized PMU-MSR writes.

Because the batched reprogram is serviced on the next VM-entry
regardless of which exit preceded it, ordinary exits -- notably the
guest's 1kHz timer tick -- absorb the cost, inflating timer
interrupts into the hundreds of microseconds. With certain workloads
this can escalate to a CSD lockup in the guest.

Add KVM_CAP_X86_DISABLE_PMU_SW_ACCOUNTING, a VM-scoped capability that
takes a bitmask of PERF_COUNT_HW_* event IDs and makes KVM skip the
software accounting of KVM-emulated instructions for those events.
KVM_CHECK_EXTENSION returns the set of events that can be disabled
(PERF_COUNT_HW_INSTRUCTIONS and PERF_COUNT_HW_BRANCH_INSTRUCTIONS --
the only events the retired-instruction path ever triggers).

The only functional change for an opted-in VM is reduced accuracy: a
guest counting instructions-retired or branches-retired undercounts by
the instructions KVM emulates in host context, i.e. the behavior that
predates the accounting cited above. Hardware-executed guest
instructions continue to be counted by the backing perf_event, and its
overflow/PMI path is unchanged. Default behavior (mask 0) is unchanged.

Assisted-by: Claude:claude-opus-4.8
Signed-off-by: Luka Absandze <absandze@amazon.de>
---
 arch/x86/include/asm/kvm_host.h |  7 +++++++
 arch/x86/kvm/pmu.c              | 21 +++++++++++++++++++++
 arch/x86/kvm/pmu.h              |  9 +++++++++
 arch/x86/kvm/x86.c              | 15 +++++++++++++++
 include/uapi/linux/kvm.h        |  1 +
 5 files changed, 53 insertions(+)

diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 5f6c1ce9673b..522f66c44e6a 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -1560,6 +1560,13 @@ struct kvm_arch {
 	bool enable_pmu;
 	bool created_mediated_pmu;
 
+	/*
+	 * Bitmask of PERF_COUNT_HW_* event IDs for which software accounting
+	 * of KVM-emulated instructions is disabled (KVM_CAP_X86_DISABLE_PMU_
+	 * SW_ACCOUNTING). See kvm_pmu_trigger_event().
+	 */
+	u64 pmu_disable_sw_accounting;
+
 	u32 notify_window;
 	u32 notify_vmexit_flags;
 	/*
diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c
index dd1c57593f48..fb3f2df068dc 100644
--- a/arch/x86/kvm/pmu.c
+++ b/arch/x86/kvm/pmu.c
@@ -1145,14 +1145,35 @@ static void kvm_pmu_trigger_event(struct kvm_vcpu *vcpu,
 	srcu_read_unlock(&vcpu->kvm->srcu, idx);
 }
 
+/*
+ * Whether software accounting of KVM-emulated instructions for @perf_hw_id is
+ * disabled via KVM_CAP_X86_DISABLE_PMU_SW_ACCOUNTING. The capability only
+ * targets the emulated (perf-based) vPMU, where the accounting drives an
+ * expensive counter reprogram; a mediated vPMU increments pmc->counter
+ * directly (and may raise a PMI on overflow), so it is never skipped.
+ */
+static bool kvm_pmu_skip_sw_accounting(struct kvm_vcpu *vcpu, u64 perf_hw_id)
+{
+	if (kvm_vcpu_has_mediated_pmu(vcpu))
+		return false;
+
+	return vcpu->kvm->arch.pmu_disable_sw_accounting & BIT_ULL(perf_hw_id);
+}
+
 void kvm_pmu_instruction_retired(struct kvm_vcpu *vcpu)
 {
+	if (kvm_pmu_skip_sw_accounting(vcpu, PERF_COUNT_HW_INSTRUCTIONS))
+		return;
+
 	kvm_pmu_trigger_event(vcpu, vcpu_to_pmu(vcpu)->pmc_counting_instructions);
 }
 EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_pmu_instruction_retired);
 
 void kvm_pmu_branch_retired(struct kvm_vcpu *vcpu)
 {
+	if (kvm_pmu_skip_sw_accounting(vcpu, PERF_COUNT_HW_BRANCH_INSTRUCTIONS))
+		return;
+
 	kvm_pmu_trigger_event(vcpu, vcpu_to_pmu(vcpu)->pmc_counting_branches);
 }
 EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_pmu_branch_retired);
diff --git a/arch/x86/kvm/pmu.h b/arch/x86/kvm/pmu.h
index a5821d7c87f9..e34b9e814355 100644
--- a/arch/x86/kvm/pmu.h
+++ b/arch/x86/kvm/pmu.h
@@ -23,6 +23,15 @@
 
 #define KVM_FIXED_PMC_BASE_IDX INTEL_PMC_IDX_FIXED
 
+/*
+ * Events for which KVM_CAP_X86_DISABLE_PMU_SW_ACCOUNTING can disable software
+ * accounting of emulated instructions. These are the only events KVM ever
+ * passes to kvm_pmu_trigger_event() (from the retired-instruction path).
+ */
+#define KVM_PMU_SW_ACCOUNTING_VALID_MASK			\
+	(BIT_ULL(PERF_COUNT_HW_INSTRUCTIONS) |			\
+	 BIT_ULL(PERF_COUNT_HW_BRANCH_INSTRUCTIONS))
+
 struct kvm_pmu_ops {
 	struct kvm_pmc *(*rdpmc_ecx_to_pmc)(struct kvm_vcpu *vcpu,
 		unsigned int idx, u64 *mask);
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index afcac1042947..cc8a36f28aea 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -4964,6 +4964,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 	case KVM_CAP_DISABLE_QUIRKS2:
 		r = kvm_caps.supported_quirks;
 		break;
+	case KVM_CAP_X86_DISABLE_PMU_SW_ACCOUNTING:
+		r = KVM_PMU_SW_ACCOUNTING_VALID_MASK;
+		break;
 	case KVM_CAP_X86_NOTIFY_VMEXIT:
 		r = kvm_caps.has_notify_vmexit;
 		break;
@@ -6730,6 +6733,18 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,
 		return -EINVAL;
 
 	switch (cap->cap) {
+	case KVM_CAP_X86_DISABLE_PMU_SW_ACCOUNTING:
+		r = -EINVAL;
+		if (cap->args[0] & ~KVM_PMU_SW_ACCOUNTING_VALID_MASK)
+			break;
+
+		mutex_lock(&kvm->lock);
+		if (!kvm->created_vcpus) {
+			kvm->arch.pmu_disable_sw_accounting = cap->args[0];
+			r = 0;
+		}
+		mutex_unlock(&kvm->lock);
+		break;
 	case KVM_CAP_DISABLE_QUIRKS2:
 		r = -EINVAL;
 		if (cap->args[0] & ~kvm_caps.supported_quirks)
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 419011097fa8..186054245b80 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -997,6 +997,7 @@ struct kvm_enable_cap {
 #define KVM_CAP_S390_KEYOP 247
 #define KVM_CAP_S390_VSIE_ESAMODE 248
 #define KVM_CAP_S390_HPAGE_2G 249
+#define KVM_CAP_X86_DISABLE_PMU_SW_ACCOUNTING 250
 
 struct kvm_irq_routing_irqchip {
 	__u32 irqchip;
-- 
2.47.3

[RFC PATCH 2/2] KVM: Documentation: Document KVM_CAP_X86_DISABLE_PMU_SW_ACCOUNTING

[Luka Absandze]

Describe the new VM-scoped capability: its parameter (a bitmask of
PERF_COUNT_HW_* event IDs), the value returned by KVM_CHECK_EXTENSION
(the set of events that can be disabled), the accuracy trade-off, and
why an emulated-vPMU host wants it.

Assisted-by: Claude:claude-opus-4.8
Signed-off-by: Luka Absandze <absandze@amazon.de>
---
 Documentation/virt/kvm/api.rst | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)

diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index a5f9ee92f43e..231a5ba7567c 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -8949,6 +8949,38 @@ enabled, cmma can't be enabled anymore and pfmfi and the storage key
 interpretation are disabled. If cmma has already been enabled or the
 hpage_2g module parameter is not set to 1, -EINVAL is returned.
 
+7.48 KVM_CAP_X86_DISABLE_PMU_SW_ACCOUNTING
+------------------------------------------
+
+:Architectures: x86
+:Type: vm
+:Parameters: args[0] - bitmask of PERF_COUNT_HW_* event IDs
+:Returns: 0 on success; -EINVAL if args[0] contains an unsupported event or
+          if the VM already has vCPUs.
+
+By default, KVM software-accounts instructions it emulates against a guest
+PMU counter programmed for retired instructions (PERF_COUNT_HW_INSTRUCTIONS)
+or retired branches (PERF_COUNT_HW_BRANCH_INSTRUCTIONS), by walking the vPMU
+counters and requesting a counter reprogram on the next VM-entry. On hosts
+with an emulated vPMU this reprogram is expensive and, under some workloads,
+inflates timer-interrupt handling enough to trigger guest CSD lockups.
+
+This capability lets userspace disable that software accounting for the given
+events. Setting a bit for an event ID makes KVM skip the accounting for that
+event: instructions KVM emulates in host context go uncounted. Hardware-
+executed guest instructions are still counted by the backing perf_event, and
+its overflow/PMI path is unaffected.
+
+The capability only affects the emulated (perf-based) vPMU. A mediated vPMU
+does not incur the reprogram cost and increments the guest counter directly,
+so accounting is never skipped for it regardless of this setting.
+
+This can only be invoked on a VM prior to the creation of vCPUs; attempting to
+set it once any vCPU exists returns -EINVAL.
+
+KVM_CHECK_EXTENSION returns the bitmask of event IDs that can be disabled.
+args[0] must be a subset of that mask.
+
 8. Other capabilities.
 ======================
 
-- 
2.47.3

Re: [RFC PATCH 1/2] KVM: x86/pmu: Add CAP to disable SW accounting of emulated instructions

[Sean Christopherson]

On Mon, Jul 20, 2026, Luka Absandze wrote:
> The only functional change for an opted-in VM is reduced accuracy: a
> guest counting instructions-retired or branches-retired undercounts by
> the instructions KVM emulates in host context, i.e. the behavior that
> predates the accounting cited above. Hardware-executed guest
> instructions continue to be counted by the backing perf_event, and its
> overflow/PMI path is unchanged.

Do you *need* per-VM control, or would a module param (or a magic value for
enable_pmu) suffice?  While I mostly buy the "it used to work this way" argument
(just "mostly", because that commit landed 4.5 years ago), I'm not exactly keen
on adding uAPI that is effectively "re-introduce a bug to workaround fundamental
design issues in KVM's emulated PMU implementation".

If we do add proper uAPI, we should extend KVM_CAP_PMU_CAPABILITY, not add a new
CAP entirely.

Re: [RFC PATCH 0/2] KVM: x86/pmu: Let userspace disable SW accounting of emulated instructions

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 769 bytes --]

On Mon, 2026-07-20 at 19:22 +0000, Luka Absandze wrote:
> On hosts that run guests with an *emulated* vPMU -- guest PMU MSR
> accesses trap into KVM and each guest counter is backed by a host
> perf_event -- the per-emulated-instruction PMU accounting added by commit
> 9cd803d496e7 ("KVM: x86: Update vPMCs when retiring instructions") is
> pathologically expensive.

Given that the guest PMU MSR accesses are trapped... is there a reason
we can't fold a *separate* count of emulated instructions into the
value we return to the guest?

I guess the overflow would come late... but isn't that tolerable? And
we could probably impose a limit on how large the addend could be —
even if we only reprogram the PMU once every 100 times, that's still a
win?

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 6179 bytes --]

Re: [RFC PATCH 1/2] KVM: x86/pmu: Add CAP to disable SW accounting of emulated instructions

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 1203 bytes --]

On Mon, 2026-07-20 at 15:27 -0700, Sean Christopherson wrote:
> On Mon, Jul 20, 2026, Luka Absandze wrote:
> > The only functional change for an opted-in VM is reduced accuracy: a
> > guest counting instructions-retired or branches-retired undercounts by
> > the instructions KVM emulates in host context, i.e. the behavior that
> > predates the accounting cited above. Hardware-executed guest
> > instructions continue to be counted by the backing perf_event, and its
> > overflow/PMI path is unchanged.
> 
> Do you *need* per-VM control, or would a module param (or a magic value for
> enable_pmu) suffice?  While I mostly buy the "it used to work this way" argument
> (just "mostly", because that commit landed 4.5 years ago), I'm not exactly keen
> on adding uAPI that is effectively "re-introduce a bug to workaround fundamental
> design issues in KVM's emulated PMU implementation".

Which is the bug? Some would argue that timer interrupts running 50
times slower is also a bug. We just get to choose *which* bug we want
the guest to experience :)

And I think that is a per-guest choice, if we can't find a way to do it
with *sufficient* correctness *and* efficiency (qv).

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 6179 bytes --]

Re: [RFC PATCH 0/2] KVM: x86/pmu: Let userspace disable SW accounting of emulated instructions

[Sean Christopherson]

On Tue, Jul 21, 2026, David Woodhouse wrote:
> On Mon, 2026-07-20 at 19:22 +0000, Luka Absandze wrote:
> > On hosts that run guests with an *emulated* vPMU -- guest PMU MSR
> > accesses trap into KVM and each guest counter is backed by a host
> > perf_event -- the per-emulated-instruction PMU accounting added by commit
> > 9cd803d496e7 ("KVM: x86: Update vPMCs when retiring instructions") is
> > pathologically expensive.
> 
> Given that the guest PMU MSR accesses are trapped... is there a reason
> we can't fold a *separate* count of emulated instructions into the
> value we return to the guest?
> 
> I guess the overflow would come late... but isn't that tolerable? 

Like ignoring emulated instructions entirely, it probably depends on the guest.

> And we could probably impose a limit on how large the addend could be — even
> if we only reprogram the PMU once every 100 times, that's still a win?

Hmm, yeah, that'd be better than ignoring emulated instruction entirely, and
would give userspace far better control over how shoddy KVM's emulation is.  Off
the cuff, the change would be fairly trivial too?  E.g.

diff --git arch/x86/kvm/pmu.c arch/x86/kvm/pmu.c
index 7f777049d328..c5f436c6f83f 100644
--- arch/x86/kvm/pmu.c
+++ arch/x86/kvm/pmu.c
@@ -1084,7 +1084,8 @@ static void kvm_pmu_incr_counter(struct kvm_pmc *pmc)
         */
        if (!kvm_vcpu_has_mediated_pmu(vcpu)) {
                pmc->emulated_counter++;
-               kvm_pmu_request_counter_reprogram(pmc);
+               if (pmc->emulated_counter > pmu_something_something_tolerance)
+                       kvm_pmu_request_counter_reprogram(pmc);
                return;
        }
 

Re: [RFC PATCH 1/2] KVM: x86/pmu: Add CAP to disable SW accounting of emulated instructions

[Sean Christopherson]

On Tue, Jul 21, 2026, David Woodhouse wrote:
> On Mon, 2026-07-20 at 15:27 -0700, Sean Christopherson wrote:
> > On Mon, Jul 20, 2026, Luka Absandze wrote:
> > > The only functional change for an opted-in VM is reduced accuracy: a
> > > guest counting instructions-retired or branches-retired undercounts by
> > > the instructions KVM emulates in host context, i.e. the behavior that
> > > predates the accounting cited above. Hardware-executed guest
> > > instructions continue to be counted by the backing perf_event, and its
> > > overflow/PMI path is unchanged.
> > 
> > Do you *need* per-VM control, or would a module param (or a magic value for
> > enable_pmu) suffice?  While I mostly buy the "it used to work this way" argument
> > (just "mostly", because that commit landed 4.5 years ago), I'm not exactly keen
> > on adding uAPI that is effectively "re-introduce a bug to workaround fundamental
> > design issues in KVM's emulated PMU implementation".
> 
> Which is the bug? Some would argue that timer interrupts running 50
> times slower is also a bug. We just get to choose *which* bug we want
> the guest to experience :)
> 
> And I think that is a per-guest choice,

Conceptually, I 100% agree.  But in practice, making a per-guest choice requires
a priori knowledge of what the guest is doing and/or what the guest needs/wants.

And so I'm asking, do your use cases have that knowledge *and* will you run VMs
with different requirements on a single host?  Because if you'll end up
configuring all VMs on a given host the same way, then I'd strongly prefer a
module param to give us more flexibility for the future, e.g. if months/years
from now we figure out a way to provide acceptable correctness and efficiency
that would allows us to drop the param entirely.

> if we can't find a way to do it with *sufficient* correctness *and*
> efficiency (qv).

Re: [RFC PATCH 0/2] KVM: x86/pmu: Let userspace disable SW accounting of emulated instructions

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 2027 bytes --]

On Tue, 2026-07-21 at 09:47 -0700, Sean Christopherson wrote:
> On Tue, Jul 21, 2026, David Woodhouse wrote:
> > On Mon, 2026-07-20 at 19:22 +0000, Luka Absandze wrote:
> > > On hosts that run guests with an *emulated* vPMU -- guest PMU MSR
> > > accesses trap into KVM and each guest counter is backed by a host
> > > perf_event -- the per-emulated-instruction PMU accounting added by commit
> > > 9cd803d496e7 ("KVM: x86: Update vPMCs when retiring instructions") is
> > > pathologically expensive.
> > 
> > Given that the guest PMU MSR accesses are trapped... is there a reason
> > we can't fold a *separate* count of emulated instructions into the
> > value we return to the guest?
> > 
> > I guess the overflow would come late... but isn't that tolerable? 
> 
> Like ignoring emulated instructions entirely, it probably depends on the guest.
> 
> > And we could probably impose a limit on how large the addend could be — even
> > if we only reprogram the PMU once every 100 times, that's still a win?
> 
> Hmm, yeah, that'd be better than ignoring emulated instruction entirely, and
> would give userspace far better control over how shoddy KVM's emulation is.  Off
> the cuff, the change would be fairly trivial too?  E.g.
> 
> diff --git arch/x86/kvm/pmu.c arch/x86/kvm/pmu.c
> index 7f777049d328..c5f436c6f83f 100644
> --- arch/x86/kvm/pmu.c
> +++ arch/x86/kvm/pmu.c
> @@ -1084,7 +1084,8 @@ static void kvm_pmu_incr_counter(struct kvm_pmc *pmc)
>          */
>         if (!kvm_vcpu_has_mediated_pmu(vcpu)) {
>                 pmc->emulated_counter++;
> -               kvm_pmu_request_counter_reprogram(pmc);
> +               if (pmc->emulated_counter > pmu_something_something_tolerance)
> +                       kvm_pmu_request_counter_reprogram(pmc);
>                 return;
>         }
> 

We also want to add the same delta when the the guest reads the
counter?

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 6179 bytes --]

Re: [RFC PATCH 0/2] KVM: x86/pmu: Let userspace disable SW accounting of emulated instructions

[Sean Christopherson]

On Tue, Jul 21, 2026, David Woodhouse wrote:
> On Tue, 2026-07-21 at 09:47 -0700, Sean Christopherson wrote:
> > On Tue, Jul 21, 2026, David Woodhouse wrote:
> > > On Mon, 2026-07-20 at 19:22 +0000, Luka Absandze wrote:
> > > > On hosts that run guests with an *emulated* vPMU -- guest PMU MSR
> > > > accesses trap into KVM and each guest counter is backed by a host
> > > > perf_event -- the per-emulated-instruction PMU accounting added by commit
> > > > 9cd803d496e7 ("KVM: x86: Update vPMCs when retiring instructions") is
> > > > pathologically expensive.
> > > 
> > > Given that the guest PMU MSR accesses are trapped... is there a reason
> > > we can't fold a *separate* count of emulated instructions into the
> > > value we return to the guest?
> > > 
> > > I guess the overflow would come late... but isn't that tolerable? 
> > 
> > Like ignoring emulated instructions entirely, it probably depends on the guest.
> > 
> > > And we could probably impose a limit on how large the addend could be — even
> > > if we only reprogram the PMU once every 100 times, that's still a win?
> > 
> > Hmm, yeah, that'd be better than ignoring emulated instruction entirely, and
> > would give userspace far better control over how shoddy KVM's emulation is.  Off
> > the cuff, the change would be fairly trivial too?  E.g.
> > 
> > diff --git arch/x86/kvm/pmu.c arch/x86/kvm/pmu.c
> > index 7f777049d328..c5f436c6f83f 100644
> > --- arch/x86/kvm/pmu.c
> > +++ arch/x86/kvm/pmu.c
> > @@ -1084,7 +1084,8 @@ static void kvm_pmu_incr_counter(struct kvm_pmc *pmc)
> >          */
> >         if (!kvm_vcpu_has_mediated_pmu(vcpu)) {
> >                 pmc->emulated_counter++;
> > -               kvm_pmu_request_counter_reprogram(pmc);
> > +               if (pmc->emulated_counter > pmu_something_something_tolerance)
> > +                       kvm_pmu_request_counter_reprogram(pmc);
> >                 return;
> >         }
> > 
> 
> We also want to add the same delta when the the guest reads the counter?

Isn't that already handled by pmc_read_counter()?  Or am I misunderstanding the
concern?

  static inline u64 pmc_read_counter(struct kvm_pmc *pmc)
  {
	u64 counter, enabled, running;

	if (kvm_vcpu_has_mediated_pmu(pmc->vcpu))
		return pmc->counter & pmc_bitmask(pmc);

	counter = pmc->counter + pmc->emulated_counter;  <===============

	if (pmc->perf_event && !pmc->is_paused)
		counter += perf_event_read_value(pmc->perf_event,
						 &enabled, &running);
	/* FIXME: Scaling needed? */
	return counter & pmc_bitmask(pmc);
  }

Re: [RFC PATCH 0/2] KVM: x86/pmu: Let userspace disable SW accounting of emulated instructions

[Absandze, Luka]

On 2026-07-21 09:47, Sean Christopherson wrote:
> Hmm, yeah, that'd be better than ignoring emulated instruction entirely, and
> would give userspace far better control over how shoddy KVM's emulation is.  Off
> the cuff, the change would be fairly trivial too?  E.g.
> 
> diff --git arch/x86/kvm/pmu.c arch/x86/kvm/pmu.c
> index 7f777049d328..c5f436c6f83f 100644
> --- arch/x86/kvm/pmu.c
> +++ arch/x86/kvm/pmu.c
> @@ -1084,7 +1084,8 @@ static void kvm_pmu_incr_counter(struct kvm_pmc *pmc)
>          */
>         if (!kvm_vcpu_has_mediated_pmu(vcpu)) {
>                 pmc->emulated_counter++;
> -               kvm_pmu_request_counter_reprogram(pmc);
> +               if (pmc->emulated_counter > pmu_something_something_tolerance)
> +                       kvm_pmu_request_counter_reprogram(pmc);
>                 return;
>         }

I believe this is fine, but perhaps the tolerance itself could be a knob
with <0 meaning no reprogram? While the contention on this path will be
much lower, we are still introducing more steal time, and the added
accuracy is probably not worth even that for guests who really only care
for the rate of change, rather than exact instruction counts.

Re: [RFC PATCH 0/2] KVM: x86/pmu: Let userspace disable SW accounting of emulated instructions

[Sean Christopherson]

On Tue, Jul 21, 2026, Luka Absandze wrote:
> On 2026-07-21 09:47, Sean Christopherson wrote:
> > Hmm, yeah, that'd be better than ignoring emulated instruction entirely, and
> > would give userspace far better control over how shoddy KVM's emulation is.  Off
> > the cuff, the change would be fairly trivial too?  E.g.
> > 
> > diff --git arch/x86/kvm/pmu.c arch/x86/kvm/pmu.c
> > index 7f777049d328..c5f436c6f83f 100644
> > --- arch/x86/kvm/pmu.c
> > +++ arch/x86/kvm/pmu.c
> > @@ -1084,7 +1084,8 @@ static void kvm_pmu_incr_counter(struct kvm_pmc *pmc)
> >          */
> >         if (!kvm_vcpu_has_mediated_pmu(vcpu)) {
> >                 pmc->emulated_counter++;
> > -               kvm_pmu_request_counter_reprogram(pmc);
> > +               if (pmc->emulated_counter > pmu_something_something_tolerance)
> > +                       kvm_pmu_request_counter_reprogram(pmc);
> >                 return;
> >         }
> 
> I believe this is fine, but perhaps the tolerance itself could be a knob

Ya, I was envisioning a knob.

> with <0 meaning no reprogram?

Eh, I don't see a need to assign special meaning to <0.  Make it a u32 (or u64),
and then a large value will provide the same behavior in practice, without needing
special handling in KVM.

> While the contention on this path will be much lower, we are still
> introducing more steal time, and the added accuracy is probably not worth
> even that for guests who really only care for the rate of change, rather than
> exact instruction counts.

Re: [RFC PATCH 1/2] KVM: x86/pmu: Add CAP to disable SW accounting of emulated instructions

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 2450 bytes --]

On Tue, 2026-07-21 at 09:53 -0700, Sean Christopherson wrote:
>  
> > Which is the bug? Some would argue that timer interrupts running 50
> > times slower is also a bug. We just get to choose *which* bug we want
> > the guest to experience :)
> > 
> > And I think that is a per-guest choice,
> 
> Conceptually, I 100% agree.  But in practice, making a per-guest choice requires
> a priori knowledge of what the guest is doing and/or what the guest needs/wants.
> 
> And so I'm asking, do your use cases have that knowledge *and* will you run VMs
> with different requirements on a single host?  Because if you'll end up
> configuring all VMs on a given host the same way, then I'd strongly prefer a
> module param to give us more flexibility for the future, e.g. if months/years
> from now we figure out a way to provide acceptable correctness and efficiency
> that would allows us to drop the param entirely.

Normally, the way we'd roll any guest-visible behavioural change out is
to preserve the existing behaviour for existing running guests. So when
we kexec to the new kernel underneath them (or when they resume from
hibernation), they get the old behaviour, and only *new* launches get
the new behaviour. Very much a per-guest thing, not a module option.

Even for things like this where we want it to reach *all* guests in the
end, that gives a relatively controllable rollout of the change — so
*if* we get complaints we can fairly quickly flip the switch so that
new launches *stop* getting the changed behaviour. Then we can think
about per-customer/per-guest opt-in/opt-out (if we really have to).

Very rarely does a module option make sense for us. Systems are largely
immutable at this scale because all else is madness. If a change like
that *is* going to be system-wide and fleet-wide, we'd be more likely
to make a one line code change to set the behaviour we want, and kexec
them all into that version.

But changing visible behaviour underneath millions of running guests
is... not good. I drink enough as it is, thank you very much...

In *this* case though, I do think we can find a middle ground that
works well enough by batching the actual updates up to a threshold.
It's not like the overflow NMI is cycle-accurate anyway; by the time
the guest has *handled* it and read the counter, it's *always* going to
be somewhat past the point at which it actually triggered, surely?

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 6179 bytes --]