From: Congkai Tan <congkai@amazon.com>
To: <oupton@kernel.org>
Cc: <blakgeof@amazon.com>, <catalin.marinas@arm.com>,
<congkai@amazon.com>, <corbet@lwn.net>, <harisokn@amazon.com>,
<joey.gouly@arm.com>, <kvm@vger.kernel.org>,
<kvmarm@lists.linux.dev>, <linux-arm-kernel@lists.infradead.org>,
<linux-doc@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
<linux-kselftest@vger.kernel.org>, <maz@kernel.org>,
<pbonzini@redhat.com>, <stanspas@amazon.de>,
<suzuki.poulose@arm.com>, <will@kernel.org>,
<yuzenghui@huawei.com>
Subject: Re: [PATCH v2 0/3] KVM: arm64: Expose PMMIR_EL1.SLOTS to guests
Date: Fri, 10 Jul 2026 06:44:23 +0000 [thread overview]
Message-ID: <20260710064423.2667799-1-congkai@amazon.com> (raw)
In-Reply-To: <ak19mPPWcupFwfuE@kernel.org>
On Tue, Jul 07, 2026 at 03:28:40PM -0700, Oliver Upton wrote:
> I've addressed all of this locally and pushed to my tree [*]. Untested,
> as always :) Would you be able to give it a spin?
Thank you so much for the review and code suggestions! I integrated
them and created v3 (with one extra change of wiring
KVM_CAP_ARM_PMU_V3_STRICT into kvm_vm_ioctl_check_extension() so
userspace can detect it). I tested the patches with kernel 7.2 and
patched QEMU 9.2.3, and the behaviors were as expected.
I'll send out the v3 patches shortly.
> Also, do you have VMM patches for using the new feature flag?
Below is the patch against QEMU 9.2.3 that I used for testing. The tests
were performed using different combinations of the four flags (host
PMMIR_EL1.SLOTS = 0xa):
- pmu-strict=on,pmu-set=on,pmmir-force=off,pmcr-n-force=off:
new happy path where guest read the correct SLOTS (0xa).
- pmu-strict=off,pmu-set=off,pmmir-force=off,pmcr-n-force=off:
old happy path where guest read SLOTS as 0x0.
- pmu-strict=off,pmu-set=on,pmmir-force=off,pmcr-n-force=off:
non-strict with a PMU set, guest read SLOTS as 0x0 (SLOTS stays
gated on the flag).
- pmu-strict=on,pmu-set=off,pmmir-force=off,pmcr-n-force=off:
strict with no PMU, so KVM_ARM_VCPU_PMU_V3_INIT failed with -ENXIO
and the guest did not boot.
- pmu-strict=on,pmu-set=on,pmmir-force=0,pmcr-n-force=off:
strict with PMMIR forced to 0 (backward compatibility), where the
KVM_SET_ONE_REG succeeded and guest read SLOTS as 0x0.
- pmu-strict=on,pmu-set=on,pmmir-force=7,pmcr-n-force=off:
strict with PMMIR forced to a mismatching value, so KVM_SET_ONE_REG
rejected with -EINVAL and guest still read SLOTS as 0xa.
- pmu-strict=off,pmu-set=on,pmmir-force=0,pmcr-n-force=off:
non-strict with PMMIR forced to 0, KVM_SET_ONE_REG returned no error
and guest read SLOTS as 0x0.
- pmu-strict=off,pmu-set=off,pmmir-force=10,pmcr-n-force=off:
non-strict with PMMIR forced to a non-zero value, KVM_SET_ONE_REG
rejected with -EINVAL and guest read SLOTS as 0x0.
- pmu-strict=off,pmu-set=off,pmmir-force=off,pmcr-n-force=3:
non-strict with a PMCR_EL0.N write (original behavior), so the write
was honored and N read back as 3.
- pmu-strict=on,pmu-set=on,pmmir-force=off,pmcr-n-force=3:
strict with a PMCR_EL0.N write, so the write was ignored and N read
back 6 (the PMU maximum).
---
Local test patch for the kernel "strict PMUv3 UAPI / PMMIR_EL1.SLOTS"
series.
Adds four 'virt' machine properties (all default off):
pmu-strict=on request KVM_ARM_VCPU_PMU_V3_STRICT at VCPU_INIT
pmu-set=off skip KVM_ARM_VCPU_PMU_V3_SET_PMU (exercise the no-PMU path)
pmmir-force=N write PMMIR_EL1=N via KVM_SET_ONE_REG (0 = set_pmmir reset)
pmcr-n-force=N write then read PMCR_EL0.N; strict ignores the write
SET_PMU is issued before PMU_V3_INIT (the kernel rejects it afterwards) and
is followed by a cpreg-cache refresh so the stale PMMIR_EL1=0 cached at init
is not written back.
Signed-off-by: Congkai Tan <congkai@amazon.com>
---
--- a/target/arm/kvm.c 2025-03-26 19:37:01.000000000 +0000
+++ b/target/arm/kvm.c 2026-07-10 04:05:16.082093995 +0000
@@ -18,11 +18,16 @@
#include "qemu/timer.h"
#include "qemu/error-report.h"
#include "qemu/main-loop.h"
+#include "qemu/cutils.h"
#include "qom/object.h"
#include "qapi/error.h"
#include "sysemu/sysemu.h"
#include "sysemu/runstate.h"
#include "sysemu/kvm.h"
+
+#ifndef KVM_ARM_VCPU_PMU_V3_STRICT
+#define KVM_ARM_VCPU_PMU_V3_STRICT 9
+#endif
#include "sysemu/kvm_int.h"
#include "kvm_arm.h"
#include "cpu.h"
@@ -1728,6 +1733,82 @@
}
}
+/* Test knob: 'pmu-strict' machine property (request KVM_ARM_VCPU_PMU_V3_STRICT). */
+bool kvm_arm_pmu_strict_enabled(void)
+{
+ Object *m = qdev_get_machine();
+
+ if (m && object_property_find(m, "pmu-strict")) {
+ return object_property_get_bool(m, "pmu-strict", NULL);
+ }
+ return false;
+}
+
+/* Test knob: 'pmu-set' machine property (issue SET_PMU; off = no-PMU path). */
+static bool kvm_arm_pmu_set_enabled(void)
+{
+ Object *m = qdev_get_machine();
+
+ if (m && object_property_find(m, "pmu-set")) {
+ return object_property_get_bool(m, "pmu-set", NULL);
+ }
+ return false;
+}
+
+/* Read the host PMUv3 perf "type" id from sysfs; -1 on failure. */
+static int kvm_arm_pmu_read_host_pmu_type(void)
+{
+ const char *path = "/sys/bus/event_source/devices/armv8_pmuv3_0/type";
+ g_autofree char *contents = NULL;
+ const char *ep;
+ gsize len;
+ int type;
+
+ if (!g_file_get_contents(path, &contents, &len, NULL)) {
+ return -1;
+ }
+ if (qemu_strtoi(contents, &ep, 10, &type) < 0 || (*ep && *ep != '\n')) {
+ return -1;
+ }
+ return type;
+}
+
+/*
+ * Select the host PMU (SET_PMU). Under the strict UAPI this is mandatory and
+ * makes the kernel snapshot PMMIR_EL1.SLOTS. Call after INIT, before PMU_V3_INIT.
+ */
+void kvm_arm_pmu_set_pmu(ARMCPU *cpu)
+{
+ int pmu_type;
+ struct kvm_device_attr attr = {
+ .group = KVM_ARM_VCPU_PMU_V3_CTRL,
+ .attr = KVM_ARM_VCPU_PMU_V3_SET_PMU,
+ };
+
+ if (!cpu->has_pmu || !kvm_arm_pmu_set_enabled()) {
+ return;
+ }
+
+ pmu_type = kvm_arm_pmu_read_host_pmu_type();
+ if (pmu_type < 0) {
+ warn_report("PMU: could not read host PMU type; "
+ "PMMIR_EL1.SLOTS will read 0 in the guest");
+ return;
+ }
+ attr.addr = (intptr_t)&pmu_type;
+
+ if (!kvm_arm_set_device_attr(cpu, &attr, "PMU SET_PMU")) {
+ warn_report("PMU: KVM_ARM_VCPU_PMU_V3_SET_PMU unsupported or failed; "
+ "PMMIR_EL1.SLOTS will read 0 in the guest");
+ return;
+ }
+
+ /* Refresh the cpreg cache so the stale PMMIR_EL1=0 isn't written back. */
+ if (!write_kvmstate_to_list(cpu)) {
+ warn_report("PMU: failed to refresh cpreg cache after SET_PMU");
+ }
+}
+
void kvm_arm_pmu_set_irq(ARMCPU *cpu, int irq)
{
struct kvm_device_attr attr = {
@@ -1904,6 +1985,10 @@
}
if (cpu->has_pmu) {
cpu->kvm_init_features[0] |= 1 << KVM_ARM_VCPU_PMU_V3;
+ /* Test knob: opt in to the strict PMUv3 UAPI. */
+ if (kvm_arm_pmu_strict_enabled()) {
+ cpu->kvm_init_features[0] |= 1 << KVM_ARM_VCPU_PMU_V3_STRICT;
+ }
}
if (cpu_isar_feature(aa64_sve, cpu)) {
assert(kvm_arm_sve_supported());
--- a/target/arm/kvm_arm.h 2025-03-26 19:37:01.000000000 +0000
+++ b/target/arm/kvm_arm.h 2026-07-10 04:05:25.331964805 +0000
@@ -207,6 +207,10 @@
void kvm_arm_pmu_init(ARMCPU *cpu);
void kvm_arm_pmu_set_irq(ARMCPU *cpu, int irq);
+/* Test knob: select the host PMU (SET_PMU). Call after INIT, before PMU_V3_INIT. */
+void kvm_arm_pmu_set_pmu(ARMCPU *cpu);
+/* Test knob: true if the 'pmu-strict' machine property is set. */
+bool kvm_arm_pmu_strict_enabled(void);
/**
* kvm_arm_pvtime_init:
@@ -280,6 +284,16 @@
g_assert_not_reached();
}
+static inline void kvm_arm_pmu_set_pmu(ARMCPU *cpu)
+{
+ g_assert_not_reached();
+}
+
+static inline bool kvm_arm_pmu_strict_enabled(void)
+{
+ return false;
+}
+
static inline void kvm_arm_pvtime_init(ARMCPU *cpu, uint64_t ipa)
{
g_assert_not_reached();
--- a/hw/arm/virt.c 2025-03-26 19:37:00.000000000 +0000
+++ b/hw/arm/virt.c 2026-07-10 04:05:51.051606007 +0000
@@ -53,6 +53,7 @@
#include "hw/loader.h"
#include "qapi/error.h"
#include "qemu/bitops.h"
+#include "qemu/cutils.h"
#include "qemu/error-report.h"
#include "qemu/module.h"
#include "hw/pci-host/gpex.h"
@@ -2070,7 +2071,64 @@
if (kvm_irqchip_in_kernel()) {
kvm_arm_pmu_set_irq(ARM_CPU(cpu), VIRTUAL_PMU_IRQ);
}
+ /* Test knob: SET_PMU before PMU_V3_INIT (see kvm_arm_pmu_set_pmu). */
+ kvm_arm_pmu_set_pmu(ARM_CPU(cpu));
kvm_arm_pmu_init(ARM_CPU(cpu));
+ /* Test knob: force PMMIR_EL1 (0 = set_pmmir() back-compat reset). */
+ if (vms->pmmir_force >= 0) {
+ uint64_t v = vms->pmmir_force;
+ int r = kvm_set_one_reg(cpu, ARM64_SYS_REG(3, 0, 9, 14, 6),
+ &v);
+ if (r) {
+ warn_report("pmmir-force: KVM_SET_ONE_REG(PMMIR_EL1) "
+ "returned %d", r);
+ }
+ /* Refresh the cpreg cache so the forced value sticks. */
+ if (!write_kvmstate_to_list(ARM_CPU(cpu))) {
+ warn_report("pmmir-force: failed to refresh cpreg "
+ "cache after KVM_SET_ONE_REG");
+ }
+ }
+ /*
+ * Test knob: SET then GET PMCR_EL0.N. strict=on -> write ignored
+ * (N stays at max); strict=off -> read-back matches the write.
+ */
+ if (vms->pmcr_n_force >= 0) {
+ uint64_t pmcr = 0;
+ int r = kvm_get_one_reg(cpu, ARM64_SYS_REG(3, 3, 9, 12, 0),
+ &pmcr);
+ if (r) {
+ warn_report("pmcr-n-force: KVM_GET_ONE_REG(PMCR_EL0) "
+ "returned %d", r);
+ } else {
+ uint64_t n = vms->pmcr_n_force;
+
+ pmcr &= ~(uint64_t)PMCRN_MASK;
+ pmcr |= (n << PMCRN_SHIFT) & PMCRN_MASK;
+ r = kvm_set_one_reg(cpu,
+ ARM64_SYS_REG(3, 3, 9, 12, 0),
+ &pmcr);
+ if (r) {
+ warn_report("pmcr-n-force: KVM_SET_ONE_REG("
+ "PMCR_EL0) returned %d", r);
+ }
+ if (!kvm_get_one_reg(cpu,
+ ARM64_SYS_REG(3, 3, 9, 12, 0),
+ &pmcr)) {
+ uint64_t got = (pmcr & PMCRN_MASK)
+ >> PMCRN_SHIFT;
+ info_report("pmcr-n-force: wrote N=%" PRIu64
+ ", read back N=%" PRIu64 " (strict=%s)",
+ n, got,
+ vms->pmu_strict ? "on" : "off");
+ }
+ /* Keep QEMU's cpreg cache consistent with the kernel. */
+ if (!write_kvmstate_to_list(ARM_CPU(cpu))) {
+ warn_report("pmcr-n-force: failed to refresh cpreg "
+ "cache after KVM_SET_ONE_REG");
+ }
+ }
+ }
}
if (steal_time) {
kvm_arm_pvtime_init(ARM_CPU(cpu), pvtime_reg_base
@@ -2676,6 +2734,88 @@
vms->mte = value;
}
+static bool virt_get_pmu_strict(Object *obj, Error **errp)
+{
+ VirtMachineState *vms = VIRT_MACHINE(obj);
+
+ return vms->pmu_strict;
+}
+
+static void virt_set_pmu_strict(Object *obj, bool value, Error **errp)
+{
+ VirtMachineState *vms = VIRT_MACHINE(obj);
+
+ vms->pmu_strict = value;
+}
+
+static bool virt_get_pmu_set(Object *obj, Error **errp)
+{
+ VirtMachineState *vms = VIRT_MACHINE(obj);
+
+ return vms->pmu_set;
+}
+
+static void virt_set_pmu_set(Object *obj, bool value, Error **errp)
+{
+ VirtMachineState *vms = VIRT_MACHINE(obj);
+
+ vms->pmu_set = value;
+}
+
+static char *virt_get_pmmir_force(Object *obj, Error **errp)
+{
+ VirtMachineState *vms = VIRT_MACHINE(obj);
+
+ if (vms->pmmir_force < 0) {
+ return g_strdup("off");
+ }
+ return g_strdup_printf("%" PRId64, vms->pmmir_force);
+}
+
+static void virt_set_pmmir_force(Object *obj, const char *value, Error **errp)
+{
+ VirtMachineState *vms = VIRT_MACHINE(obj);
+ uint64_t v;
+
+ if (!strcmp(value, "off")) {
+ vms->pmmir_force = -1;
+ return;
+ }
+ if (qemu_strtou64(value, NULL, 0, &v) < 0) {
+ error_setg(errp, "Invalid pmmir-force value '%s' "
+ "(expected 'off' or an integer)", value);
+ return;
+ }
+ vms->pmmir_force = (int64_t)v;
+}
+
+static char *virt_get_pmcr_n_force(Object *obj, Error **errp)
+{
+ VirtMachineState *vms = VIRT_MACHINE(obj);
+
+ if (vms->pmcr_n_force < 0) {
+ return g_strdup("off");
+ }
+ return g_strdup_printf("%" PRId64, vms->pmcr_n_force);
+}
+
+static void virt_set_pmcr_n_force(Object *obj, const char *value, Error **errp)
+{
+ VirtMachineState *vms = VIRT_MACHINE(obj);
+ uint64_t v;
+
+ if (!strcmp(value, "off")) {
+ vms->pmcr_n_force = -1;
+ return;
+ }
+ if (qemu_strtou64(value, NULL, 0, &v) < 0) {
+ error_setg(errp, "Invalid pmcr-n-force value '%s' "
+ "(expected 'off' or an integer)", value);
+ return;
+ }
+ vms->pmcr_n_force = (int64_t)v;
+}
+
static char *virt_get_gic_version(Object *obj, Error **errp)
{
VirtMachineState *vms = VIRT_MACHINE(obj);
@@ -3233,6 +3373,41 @@
"to a KVM guest using ACPI and guest external abort exceptions");
object_class_property_add_bool(oc, "mte", virt_get_mte, virt_set_mte);
+
+ object_class_property_add_bool(oc, "pmu-strict", virt_get_pmu_strict,
+ virt_set_pmu_strict);
+ object_class_property_set_description(oc, "pmu-strict",
+ "Test knob: request the strict "
+ "PMUv3 UAPI (KVM_ARM_VCPU_PMU_V3_"
+ "STRICT) for each vCPU");
+
+ object_class_property_add_bool(oc, "pmu-set", virt_get_pmu_set,
+ virt_set_pmu_set);
+ object_class_property_set_description(oc, "pmu-set",
+ "Test knob: issue "
+ "KVM_ARM_VCPU_PMU_V3_SET_PMU "
+ "(default on); off exercises the "
+ "strict no-PMU path");
+
+ object_class_property_add_str(oc, "pmmir-force", virt_get_pmmir_force,
+ virt_set_pmmir_force);
+ object_class_property_set_description(oc, "pmmir-force",
+ "Test knob: force PMMIR_EL1 to this "
+ "value via KVM_SET_ONE_REG after the "
+ "PMU is selected. 'off' (default) "
+ "leaves the kernel snapshot intact; "
+ "0 exercises the set_pmmir() "
+ "back-compat reset");
+
+ object_class_property_add_str(oc, "pmcr-n-force", virt_get_pmcr_n_force,
+ virt_set_pmcr_n_force);
+ object_class_property_set_description(oc, "pmcr-n-force",
+ "Test knob: write PMCR_EL0.N with "
+ "this value via KVM_SET_ONE_REG, then "
+ "read it back and log the result. "
+ "Under pmu-strict=on the kernel "
+ "ignores the write (N stays at max); "
+ "off (default) disables the knob");
object_class_property_set_description(oc, "mte",
"Set on/off to enable/disable emulating a "
"guest CPU which implements the ARM "
@@ -3290,6 +3465,12 @@
/* EL2 is also disabled by default, for similar reasons */
vms->virt = false;
+ /* Test knobs default: select a PMU, no strict flag, no pmmir override */
+ vms->pmu_strict = false;
+ vms->pmu_set = true;
+ vms->pmmir_force = -1;
+ vms->pmcr_n_force = -1;
+
/* High memory is enabled by default */
vms->highmem = true;
vms->highmem_compact = !vmc->no_highmem_compact;
--- a/include/hw/arm/virt.h 2025-03-26 19:37:00.000000000 +0000
+++ b/include/hw/arm/virt.h 2026-07-10 04:06:12.091312904 +0000
@@ -180,6 +180,11 @@
char *oem_id;
char *oem_table_id;
bool ns_el2_virt_timer_irq;
+ /* Test knobs for the strict PMUv3 series (-1 / false = off). */
+ bool pmu_strict; /* request KVM_ARM_VCPU_PMU_V3_STRICT */
+ bool pmu_set; /* issue SET_PMU (default on) */
+ int64_t pmmir_force; /* force PMMIR_EL1 via KVM_SET_ONE_REG */
+ int64_t pmcr_n_force; /* write+read PMCR_EL0.N via KVM_SET_ONE_REG */
};
#define VIRT_ECAM_ID(high) (high ? VIRT_HIGH_PCIE_ECAM : VIRT_PCIE_ECAM)
prev parent reply other threads:[~2026-07-10 6:45 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-07-02 19:04 [PATCH v2 0/3] KVM: arm64: Expose PMMIR_EL1.SLOTS to guests Congkai Tan
2026-07-02 19:04 ` [PATCH v2 1/3] KVM: arm64: Add KVM_ARM_VCPU_PMU_V3_STRICT vCPU feature Congkai Tan
2026-07-02 19:29 ` sashiko-bot
2026-07-02 19:04 ` [PATCH v2 2/3] KVM: arm64: Expose PMMIR_EL1.SLOTS under strict PMUv3 UAPI Congkai Tan
2026-07-02 19:04 ` [PATCH v2 3/3] KVM: arm64: Advertise STALL_SLOT* in PMCEID1 " Congkai Tan
2026-07-07 22:28 ` [PATCH v2 0/3] KVM: arm64: Expose PMMIR_EL1.SLOTS to guests Oliver Upton
2026-07-10 6:44 ` Congkai Tan [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260710064423.2667799-1-congkai@amazon.com \
--to=congkai@amazon.com \
--cc=blakgeof@amazon.com \
--cc=catalin.marinas@arm.com \
--cc=corbet@lwn.net \
--cc=harisokn@amazon.com \
--cc=joey.gouly@arm.com \
--cc=kvm@vger.kernel.org \
--cc=kvmarm@lists.linux.dev \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-kselftest@vger.kernel.org \
--cc=maz@kernel.org \
--cc=oupton@kernel.org \
--cc=pbonzini@redhat.com \
--cc=stanspas@amazon.de \
--cc=suzuki.poulose@arm.com \
--cc=will@kernel.org \
--cc=yuzenghui@huawei.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.