* [PATCH v4 0/2] KVM: arm64: Clear VM-wide state when kvm_setup_vcpu() fails @ 2026-11-02 7:58 Fuad Tabba 2026-11-02 7:58 ` [PATCH v4 1/2] KVM: arm64: Clear the VM's feature bitmap " Fuad Tabba 2026-11-02 7:58 ` [PATCH v4 2/2] KVM: arm64: Clear the default PMU " Fuad Tabba 0 siblings, 2 replies; 7+ messages in thread From: Fuad Tabba @ 2026-11-02 7:58 UTC (permalink / raw) To: Marc Zyngier, Oliver Upton Cc: Joey Gouly, Steffen Eiden, Suzuki K Poulose, Zenghui Yu, Will Deacon, Lorenzo Stoakes, Jack Thomson, Congkai Tan, kvmarm, linux-arm-kernel, linux-kernel, Fuad Tabba Hi folks, Changes since v3 [1]: - New patch 2, clearing the default PMU left set by a failed INIT. - Reworded patch 1's commit message. - Rebased on 7.4-rc1. A KVM_ARM_VCPU_INIT whose kvm_setup_vcpu() fails doesn't undo its changes to VM-wide state. Patch 1 clears the feature bitmap, which the PMU attributes read before the vCPU is initialized. Patch 2 clears the default PMU, which a retry with KVM_ARM_VCPU_PMU_V3_STRICT would otherwise inherit. Tested under QEMU on a host without FEAT_NV1 (an Apple M2 MIDR, for which has_nv1() returns false), with a reproducer that fails an INIT and then initializes the PMU, directly for patch 1 and after a STRICT retry for patch 2. Based on Linux 7.4-rc1. Cheers, /fuad [1] https://lore.kernel.org/all/20260921190843.107881-1-fuad.tabba@linux.dev/ Fuad Tabba (2): KVM: arm64: Clear the VM's feature bitmap when kvm_setup_vcpu() fails KVM: arm64: Clear the default PMU when kvm_setup_vcpu() fails arch/arm64/kvm/arm.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) base-commit: 6628e7d4c6b25c8d18f1345e3a20d21829aae7e4 -- 2.47.3 ^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH v4 1/2] KVM: arm64: Clear the VM's feature bitmap when kvm_setup_vcpu() fails 2026-11-02 7:58 [PATCH v4 0/2] KVM: arm64: Clear VM-wide state when kvm_setup_vcpu() fails Fuad Tabba @ 2026-11-02 7:58 ` Fuad Tabba 2026-11-02 9:24 ` Suzuki K Poulose 2026-11-02 7:58 ` [PATCH v4 2/2] KVM: arm64: Clear the default PMU " Fuad Tabba 1 sibling, 1 reply; 7+ messages in thread From: Fuad Tabba @ 2026-11-02 7:58 UTC (permalink / raw) To: Marc Zyngier, Oliver Upton Cc: Joey Gouly, Steffen Eiden, Suzuki K Poulose, Zenghui Yu, Will Deacon, Lorenzo Stoakes, Jack Thomson, Congkai Tan, kvmarm, linux-arm-kernel, linux-kernel, Fuad Tabba KVM_ARM_VCPU_INIT records the requested features in the VM-wide bitmap before kvm_setup_vcpu() runs, and leaves them there when setup fails (e.g., HAS_EL2_E2H0 on a host without FEAT_NV1). The PMU attributes read that bitmap before the vCPU is initialized, so after such a failure with PMU_V3 requested, KVM_SET_DEVICE_ATTR on the PMU group no longer returns -ENODEV. Until an INIT has succeeded, every INIT that gets past the feature checks rewrites the bitmap. After that, kvm_vcpu_init_changed() requires the requested features to match it, so the copy is a no-op. Clear the bitmap on failure while KVM_ARCH_FLAG_VCPU_FEATURES_CONFIGURED is clear. Fixes: 427733579744e ("KVM: arm64: Select default PMU in KVM_ARM_VCPU_INIT handler") Suggested-by: Oliver Upton <oupton@kernel.org> Reviewed-by: Lorenzo Stoakes (ARM) <ljs@kernel.org> Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev> --- arch/arm64/kvm/arm.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 157bf4b812020..3e91e01b032c4 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1856,8 +1856,13 @@ static int __kvm_vcpu_set_target(struct kvm_vcpu *vcpu, bitmap_copy(kvm->arch.vcpu_features, &features, KVM_VCPU_MAX_FEATURES); ret = kvm_setup_vcpu(vcpu); - if (ret) + if (ret) { + /* Clear the bitmap if setup fails on the first vCPU to be initialized. */ + if (!test_bit(KVM_ARCH_FLAG_VCPU_FEATURES_CONFIGURED, &kvm->arch.flags)) + bitmap_zero(kvm->arch.vcpu_features, KVM_VCPU_MAX_FEATURES); + goto out_unlock; + } /* Now we know what it is, we can reset it. */ kvm_reset_vcpu(vcpu); -- 2.47.3 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH v4 1/2] KVM: arm64: Clear the VM's feature bitmap when kvm_setup_vcpu() fails 2026-11-02 7:58 ` [PATCH v4 1/2] KVM: arm64: Clear the VM's feature bitmap " Fuad Tabba @ 2026-11-02 9:24 ` Suzuki K Poulose 2026-11-02 9:42 ` Fuad Tabba 0 siblings, 1 reply; 7+ messages in thread From: Suzuki K Poulose @ 2026-11-02 9:24 UTC (permalink / raw) To: Fuad Tabba, Marc Zyngier, Oliver Upton Cc: Joey Gouly, Steffen Eiden, Zenghui Yu, Will Deacon, Lorenzo Stoakes, Jack Thomson, Congkai Tan, kvmarm, linux-arm-kernel, linux-kernel, Fuad Tabba On 02/11/2026 07:58, Fuad Tabba wrote: > KVM_ARM_VCPU_INIT records the requested features in the VM-wide bitmap > before kvm_setup_vcpu() runs, and leaves them there when setup fails > (e.g., HAS_EL2_E2H0 on a host without FEAT_NV1). The PMU attributes > read that bitmap before the vCPU is initialized, so after such a > failure with PMU_V3 requested, KVM_SET_DEVICE_ATTR on the PMU group no > longer returns -ENODEV. > > Until an INIT has succeeded, every INIT that gets past the feature > checks rewrites the bitmap. After that, kvm_vcpu_init_changed() > requires the requested features to match it, so the copy is a no-op. > Clear the bitmap on failure while KVM_ARCH_FLAG_VCPU_FEATURES_CONFIGURED > is clear. > > Fixes: 427733579744e ("KVM: arm64: Select default PMU in KVM_ARM_VCPU_INIT handler") > Suggested-by: Oliver Upton <oupton@kernel.org> > Reviewed-by: Lorenzo Stoakes (ARM) <ljs@kernel.org> > Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev> > --- > arch/arm64/kvm/arm.c | 7 ++++++- > 1 file changed, 6 insertions(+), 1 deletion(-) > > diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c > index 157bf4b812020..3e91e01b032c4 100644 > --- a/arch/arm64/kvm/arm.c > +++ b/arch/arm64/kvm/arm.c > @@ -1856,8 +1856,13 @@ static int __kvm_vcpu_set_target(struct kvm_vcpu *vcpu, > bitmap_copy(kvm->arch.vcpu_features, &features, KVM_VCPU_MAX_FEATURES); > > ret = kvm_setup_vcpu(vcpu); > - if (ret) > + if (ret) { > + /* Clear the bitmap if setup fails on the first vCPU to be initialized. */ > + if (!test_bit(KVM_ARCH_FLAG_VCPU_FEATURES_CONFIGURED, &kvm->arch.flags)) Is this check necessary ? We go into kvm_setup_vcpu() only if the bit is cleared and we do this under the config_lock mutex. The flag is only set after we have finished the kvm_setup_vcpu() successfully, later down. Otherwise, Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com> > + bitmap_zero(kvm->arch.vcpu_features, KVM_VCPU_MAX_FEATURES); > + > goto out_unlock; > + } > > /* Now we know what it is, we can reset it. */ > kvm_reset_vcpu(vcpu); ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH v4 1/2] KVM: arm64: Clear the VM's feature bitmap when kvm_setup_vcpu() fails 2026-11-02 9:24 ` Suzuki K Poulose @ 2026-11-02 9:42 ` Fuad Tabba 2026-11-02 9:48 ` Suzuki K Poulose 0 siblings, 1 reply; 7+ messages in thread From: Fuad Tabba @ 2026-11-02 9:42 UTC (permalink / raw) To: Suzuki K Poulose Cc: Marc Zyngier, Oliver Upton, Joey Gouly, Steffen Eiden, Zenghui Yu, Will Deacon, Lorenzo Stoakes, Jack Thomson, Congkai Tan, kvmarm, linux-arm-kernel, linux-kernel Hi Suzuki, On Mon, 02 Nov 2026 09:24:15 +0000, Suzuki K Poulose <suzuki.poulose@arm.com> wrote: [...] > > diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c [...] > > @@ -1856,8 +1856,13 @@ static int __kvm_vcpu_set_target(struct kvm_vcpu *vcpu, > > bitmap_copy(kvm->arch.vcpu_features, &features, KVM_VCPU_MAX_FEATURES); > > > > ret = kvm_setup_vcpu(vcpu); > > - if (ret) > > + if (ret) { > > + /* Clear the bitmap if setup fails on the first vCPU to be initialized. */ > > + if (!test_bit(KVM_ARCH_FLAG_VCPU_FEATURES_CONFIGURED, &kvm->arch.flags)) > > Is this check necessary ? We go into kvm_setup_vcpu() only if the bit is > cleared and we do this under the config_lock mutex. The flag is only set > after we have finished the kvm_setup_vcpu() successfully, later down. It is. An INIT on a second vCPU with the same features also calls kvm_setup_vcpu(), with the flag already set, and kvm_vcpu_init_nested() can return -ENOMEM there. Thanks, /fuad ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH v4 1/2] KVM: arm64: Clear the VM's feature bitmap when kvm_setup_vcpu() fails 2026-11-02 9:42 ` Fuad Tabba @ 2026-11-02 9:48 ` Suzuki K Poulose 0 siblings, 0 replies; 7+ messages in thread From: Suzuki K Poulose @ 2026-11-02 9:48 UTC (permalink / raw) To: Fuad Tabba Cc: Marc Zyngier, Oliver Upton, Joey Gouly, Steffen Eiden, Zenghui Yu, Will Deacon, Lorenzo Stoakes, Jack Thomson, Congkai Tan, kvmarm, linux-arm-kernel, linux-kernel On 02/11/2026 09:42, Fuad Tabba wrote: > Hi Suzuki, > > On Mon, 02 Nov 2026 09:24:15 +0000, Suzuki K Poulose > <suzuki.poulose@arm.com> wrote: > [...] >>> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c > [...] >>> @@ -1856,8 +1856,13 @@ static int __kvm_vcpu_set_target(struct kvm_vcpu *vcpu, >>> bitmap_copy(kvm->arch.vcpu_features, &features, KVM_VCPU_MAX_FEATURES); >>> >>> ret = kvm_setup_vcpu(vcpu); >>> - if (ret) >>> + if (ret) { >>> + /* Clear the bitmap if setup fails on the first vCPU to be initialized. */ >>> + if (!test_bit(KVM_ARCH_FLAG_VCPU_FEATURES_CONFIGURED, &kvm->arch.flags)) >> >> Is this check necessary ? We go into kvm_setup_vcpu() only if the bit is >> cleared and we do this under the config_lock mutex. The flag is only set >> after we have finished the kvm_setup_vcpu() successfully, later down. > > It is. An INIT on a second vCPU with the same features also calls > kvm_setup_vcpu(), with the flag already set, and > kvm_vcpu_init_nested() can return -ENOMEM there. Sorry, my bad. I missed the second condition after the flag check in. Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com> /me Where's coffee! > > Thanks, > /fuad ^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH v4 2/2] KVM: arm64: Clear the default PMU when kvm_setup_vcpu() fails 2026-11-02 7:58 [PATCH v4 0/2] KVM: arm64: Clear VM-wide state when kvm_setup_vcpu() fails Fuad Tabba 2026-11-02 7:58 ` [PATCH v4 1/2] KVM: arm64: Clear the VM's feature bitmap " Fuad Tabba @ 2026-11-02 7:58 ` Fuad Tabba 2026-11-02 9:56 ` Suzuki K Poulose 1 sibling, 1 reply; 7+ messages in thread From: Fuad Tabba @ 2026-11-02 7:58 UTC (permalink / raw) To: Marc Zyngier, Oliver Upton Cc: Joey Gouly, Steffen Eiden, Suzuki K Poulose, Zenghui Yu, Will Deacon, Lorenzo Stoakes, Jack Thomson, Congkai Tan, kvmarm, linux-arm-kernel, linux-kernel, Fuad Tabba Under KVM_ARM_VCPU_PMU_V3_STRICT, the VM has no PMU until userspace selects one, and initializing the PMU fails with -ENXIO until then. However, without STRICT, kvm_setup_vcpu() selects the default PMU before it prepares the vCPU for nested virtualization, and leaves it set when that preparation fails. After a retry with STRICT, initializing the PMU succeeds and the guest gets the PMU of the CPU the failed INIT ran on. Drop the PMU and its number of counters along with the VM's feature bitmap. While KVM_ARCH_FLAG_VCPU_FEATURES_CONFIGURED is clear, no vCPU has been initialized and the PMU attributes return -ENODEV, so only such a failure can have set them. Once the flag is set, every INIT has the features of the first one that succeeded, which either selected a PMU or had none to select, so kvm_setup_vcpu() doesn't select another. Fixes: ad220e275c398 ("KVM: arm64: Add KVM_ARM_VCPU_PMU_V3_STRICT vCPU feature") Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev> --- arch/arm64/kvm/arm.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 3e91e01b032c4..65194e02cdbc2 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1857,9 +1857,12 @@ static int __kvm_vcpu_set_target(struct kvm_vcpu *vcpu, ret = kvm_setup_vcpu(vcpu); if (ret) { - /* Clear the bitmap if setup fails on the first vCPU to be initialized. */ - if (!test_bit(KVM_ARCH_FLAG_VCPU_FEATURES_CONFIGURED, &kvm->arch.flags)) + /* Clear the bitmap and PMU if setup fails on the first vCPU to be initialized. */ + if (!test_bit(KVM_ARCH_FLAG_VCPU_FEATURES_CONFIGURED, &kvm->arch.flags)) { bitmap_zero(kvm->arch.vcpu_features, KVM_VCPU_MAX_FEATURES); + kvm->arch.arm_pmu = NULL; + kvm->arch.nr_pmu_counters = 0; + } goto out_unlock; } -- 2.47.3 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH v4 2/2] KVM: arm64: Clear the default PMU when kvm_setup_vcpu() fails 2026-11-02 7:58 ` [PATCH v4 2/2] KVM: arm64: Clear the default PMU " Fuad Tabba @ 2026-11-02 9:56 ` Suzuki K Poulose 0 siblings, 0 replies; 7+ messages in thread From: Suzuki K Poulose @ 2026-11-02 9:56 UTC (permalink / raw) To: Fuad Tabba, Marc Zyngier, Oliver Upton Cc: Joey Gouly, Steffen Eiden, Zenghui Yu, Will Deacon, Lorenzo Stoakes, Jack Thomson, Congkai Tan, kvmarm, linux-arm-kernel, linux-kernel, Fuad Tabba On 02/11/2026 07:58, Fuad Tabba wrote: > Under KVM_ARM_VCPU_PMU_V3_STRICT, the VM has no PMU until userspace > selects one, and initializing the PMU fails with -ENXIO until then. > However, without STRICT, kvm_setup_vcpu() selects the default PMU > before it prepares the vCPU for nested virtualization, and leaves it > set when that preparation fails. After a retry with STRICT, > initializing the PMU succeeds and the guest gets the PMU of the CPU > the failed INIT ran on. > > Drop the PMU and its number of counters along with the VM's feature > bitmap. While KVM_ARCH_FLAG_VCPU_FEATURES_CONFIGURED is clear, no vCPU > has been initialized and the PMU attributes return -ENODEV, so only > such a failure can have set them. Once the flag is set, every INIT has the > features of the first one that succeeded, which either selected a PMU > or had none to select, so kvm_setup_vcpu() doesn't select another. > > Fixes: ad220e275c398 ("KVM: arm64: Add KVM_ARM_VCPU_PMU_V3_STRICT vCPU feature") > Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev> > --- > arch/arm64/kvm/arm.c | 7 +++++-- > 1 file changed, 5 insertions(+), 2 deletions(-) > > diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c > index 3e91e01b032c4..65194e02cdbc2 100644 > --- a/arch/arm64/kvm/arm.c > +++ b/arch/arm64/kvm/arm.c > @@ -1857,9 +1857,12 @@ static int __kvm_vcpu_set_target(struct kvm_vcpu *vcpu, > > ret = kvm_setup_vcpu(vcpu); > if (ret) { > - /* Clear the bitmap if setup fails on the first vCPU to be initialized. */ > - if (!test_bit(KVM_ARCH_FLAG_VCPU_FEATURES_CONFIGURED, &kvm->arch.flags)) > + /* Clear the bitmap and PMU if setup fails on the first vCPU to be initialized. */ > + if (!test_bit(KVM_ARCH_FLAG_VCPU_FEATURES_CONFIGURED, &kvm->arch.flags)) { > bitmap_zero(kvm->arch.vcpu_features, KVM_VCPU_MAX_FEATURES); > + kvm->arch.arm_pmu = NULL; > + kvm->arch.nr_pmu_counters = 0; > + } minor nit: While the above works, it looks a bit too much extracting the PMU details to arm.c. Alternatively we could do something like this: diff --git a/arch/arm64/kvm/pmu-emul.c b/arch/arm64/kvm/pmu-emul.c index e2b28dbc0451..7a669cccf912 100644 --- a/arch/arm64/kvm/pmu-emul.c +++ b/arch/arm64/kvm/pmu-emul.c @@ -1088,6 +1088,11 @@ int kvm_arm_set_default_pmu(struct kvm *kvm) return 0; } +int kvm_arm_clear_default_pmu(struct kvm *kvm) +{ + kvm_arm_set_pmu(kvm, NULL); +} + diff --git a/include/kvm/arm_pmu.h b/include/kvm/arm_pmu.h index 6b4a118d17ca..571c0ba8507c 100644 --- a/include/kvm/arm_pmu.h +++ b/include/kvm/arm_pmu.h @@ -92,6 +92,7 @@ void kvm_vcpu_pmu_resync_el0(void); u8 kvm_arm_pmu_get_pmuver_limit(void); u64 kvm_pmu_evtyper_mask(struct kvm *kvm); int kvm_arm_set_default_pmu(struct kvm *kvm); +int kvm_arm_clear_default_pmu(struct kvm *kvm); u8 kvm_arm_pmu_get_max_counters(struct kvm *kvm); u64 kvm_vcpu_read_pmcr(struct kvm_vcpu *vcpu); Either ways, Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com ^ permalink raw reply related [flat|nested] 7+ messages in thread
end of thread, other threads:[~2026-11-02 9:57 UTC | newest] Thread overview: 7+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2026-11-02 7:58 [PATCH v4 0/2] KVM: arm64: Clear VM-wide state when kvm_setup_vcpu() fails Fuad Tabba 2026-11-02 7:58 ` [PATCH v4 1/2] KVM: arm64: Clear the VM's feature bitmap " Fuad Tabba 2026-11-02 9:24 ` Suzuki K Poulose 2026-11-02 9:42 ` Fuad Tabba 2026-11-02 9:48 ` Suzuki K Poulose 2026-11-02 7:58 ` [PATCH v4 2/2] KVM: arm64: Clear the default PMU " Fuad Tabba 2026-11-02 9:56 ` Suzuki K Poulose
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox