[PATCH v2 0/2] KVM: arm64: Fix MPIDR lookup for unreset vCPUs

[PATCH v2 0/2] KVM: arm64: Fix MPIDR lookup for unreset vCPUs

[fuqiang wang]

From: wangfuqiang49 <wangfuqiang49@jd.com>

Hi,

This series fixes an MPIDR lookup issue when a VM is created with CPU
hotplug support.

kvm_init_mpidr_data() builds a compressed MPIDR-to-vCPU lookup table by
walking all possible vCPUs. However, vCPUs that have not been reset yet
still have a zero MPIDR_EL1 state, which aliases vCPU0. This can cause
cmpidr_to_idx[0] to be overwritten with the index of an unreset vCPU.

As a result, MPIDR 0 lookups can return the wrong vCPU, preventing
interrupts targeting vCPU0 from being delivered correctly and making
guest boot extremely slow in configurations using CPU hotplug.

Patch 1 adds named definitions for the MPIDR_EL1 RES1 bit used to detect
whether a vCPU's MPIDR_EL1 state has been reset.
Patch 2 skips unreset vCPUs when populating the compressed MPIDR lookup
table.

Thanks,
Fuqiang

SELF-TEST result:

1. use crash debug cmpidr_to_idx[]

(qemu args: -smp maxcpus=512,cpus=4,sockets=2,cores=128,threads=2)

crash> struct kvm_mpidr_data.mpidr_mask 0xffff2020acda15e0
  mpidr_mask = 3,
crash> x/8xh 0xffff2020acda15e8
0xffff2020acda15e8:     0x0000  0x0001  0x0002  0x0003  0x642f  0x7665  0x6e2f  0x6d76

2. use link [1] to test IPI latency(Before the patch was merged, it often took
   seconds for a kernel thread to be scheduled after being woken up.)

IPI Latency Matrix - avg (ns) [100000 samples]

          CPU0     CPU1     CPU2     CPU3
CPU0           -    1355    2242    2238
CPU1        2730       -    2391    1852
CPU2        2149    3015       -    2033
CPU3        2005    1651    1801       -


Changes in v2:
- Fix macro naming/placement/type and pass vcpu instead of pre-masked
  affinity to correctly check MPIDR_EL1 bit[31] (sashiko bot)

v1: https://lore.kernel.org/all/20260611090934.84469-1-fuqiang.wng@gmail.com/

[1]: https://github.com/cai-fuqiang/kernel_test/tree/master/ipi_latency

fuqiang wang (2):
  arm64: Add MPIDR_EL1 RES1 definitions
  KVM: arm64: Skip unreset vCPUs in MPIDR lookup table

 arch/arm64/include/asm/cputype.h     |  1 +
 arch/arm64/include/asm/kvm_emulate.h |  9 +++++++++
 arch/arm64/kvm/arm.c                 | 14 ++++++++++++--
 3 files changed, 22 insertions(+), 2 deletions(-)

-- 
2.47.0

[PATCH v2 1/2] arm64: Add MPIDR_EL1 RES1 definitions

[fuqiang wang]

MPIDR_EL1 bit 31 is RES1. Add named definitions for the bit.

Signed-off-by: fuqiang wang <fuqiang.wng@gmail.com>
---
 arch/arm64/include/asm/cputype.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/arm64/include/asm/cputype.h b/arch/arm64/include/asm/cputype.h
index 7b518e81dd15..22dc44d2de03 100644
--- a/arch/arm64/include/asm/cputype.h
+++ b/arch/arm64/include/asm/cputype.h
@@ -9,6 +9,7 @@
 
 #define MPIDR_UP_BITMASK	(0x1 << 30)
 #define MPIDR_MT_BITMASK	(0x1 << 24)
+#define MPIDR_RES1_BITMASK	(UL(1) << 31)
 #define MPIDR_HWID_BITMASK	UL(0xff00ffffff)
 
 #define MPIDR_LEVEL_BITS_SHIFT	3
-- 
2.47.0

[PATCH v2 2/2] KVM: arm64: Skip unreset vCPUs in MPIDR lookup table

[fuqiang wang]

When a VM is created with CPU hotplug support, kvm_for_each_vcpu() can
walk vCPUs that have not been reset yet. Their MPIDR_EL1 state is still
zero, which aliases vCPU0 when populating the compressed MPIDR lookup
table.

As a result, cmpidr_to_idx[0] can be overwritten with the index of an
unreset vCPU. A lookup for MPIDR 0 then returns the wrong vCPU, which
can prevent interrupts targeting vCPU0 from being delivered correctly
and make guest boot extremely slow.

Skip vCPUs whose MPIDR_EL1 value does not have the RES1 bit set.

Fixes: 5544750efd51 ("KVM: arm64: Build MPIDR to vcpu index cache at runtime")
Signed-off-by: fuqiang wang <fuqiang.wng@gmail.com>
---
 arch/arm64/include/asm/kvm_emulate.h |  9 +++++++++
 arch/arm64/kvm/arm.c                 | 14 ++++++++++++--
 2 files changed, 21 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
index 5bf3d7e1d92c..3d2f0a8b040d 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -506,6 +506,15 @@ static inline unsigned long kvm_vcpu_get_mpidr_aff(struct kvm_vcpu *vcpu)
 	return __vcpu_sys_reg(vcpu, MPIDR_EL1) & MPIDR_HWID_BITMASK;
 }
 
+/*
+ * Check if vCPU MPIDR_EL1 has been reset. MPIDR_EL1.bit[31] is RES1
+ * and set during reset; a zero value indicates the vCPU is unreset.
+ */
+static inline bool kvm_vcpu_mpidr_is_reset(struct kvm_vcpu *vcpu)
+{
+	return !!(__vcpu_sys_reg(vcpu, MPIDR_EL1) & MPIDR_RES1_BITMASK);
+}
+
 static inline void kvm_vcpu_set_be(struct kvm_vcpu *vcpu)
 {
 	if (vcpu_mode_is_32bit(vcpu)) {
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 3732ee9eb0d4..fccfa97370df 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -887,8 +887,18 @@ static void kvm_init_mpidr_data(struct kvm *kvm)
 	data->mpidr_mask = mask;
 
 	kvm_for_each_vcpu(c, vcpu, kvm) {
-		u64 aff = kvm_vcpu_get_mpidr_aff(vcpu);
-		u16 index = kvm_mpidr_index(data, aff);
+		u64 aff;
+		u16 index;
+
+		/*
+		 * Skip vCPUs that haven't been reset yet; their MPIDR_EL1 is
+		 * zero.
+		 */
+		if (!kvm_vcpu_mpidr_is_reset(vcpu))
+			continue;
+
+		aff = kvm_vcpu_get_mpidr_aff(vcpu);
+		index = kvm_mpidr_index(data, aff);
 
 		data->cmpidr_to_idx[index] = c;
 	}
-- 
2.47.0

Re: [PATCH v2 2/2] KVM: arm64: Skip unreset vCPUs in MPIDR lookup table

[Marc Zyngier]

On Thu, 11 Jun 2026 15:40:42 +0100,
fuqiang wang <fuqiang.wng@gmail.com> wrote:
> 
> When a VM is created with CPU hotplug support,

I'm not sure what you mean by "hotplug support". KVM has no support
for vcpu hotplug at all (all vcpus must have been created before the
VM can run).

> kvm_for_each_vcpu() can
> walk vCPUs that have not been reset yet. Their MPIDR_EL1 state is still
> zero, which aliases vCPU0 when populating the compressed MPIDR lookup
> table.
> 
> As a result, cmpidr_to_idx[0] can be overwritten with the index of an
> unreset vCPU. A lookup for MPIDR 0 then returns the wrong vCPU, which
> can prevent interrupts targeting vCPU0 from being delivered correctly
> and make guest boot extremely slow.

OK, so the problem you are describing is related to vcpus that have
haven't gone through the INIT phase, and really has nothing to do with
hotplug.

> 
> Skip vCPUs whose MPIDR_EL1 value does not have the RES1 bit set.
> 
> Fixes: 5544750efd51 ("KVM: arm64: Build MPIDR to vcpu index cache at runtime")
> Signed-off-by: fuqiang wang <fuqiang.wng@gmail.com>
> ---
>  arch/arm64/include/asm/kvm_emulate.h |  9 +++++++++
>  arch/arm64/kvm/arm.c                 | 14 ++++++++++++--
>  2 files changed, 21 insertions(+), 2 deletions(-)
> 
> diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
> index 5bf3d7e1d92c..3d2f0a8b040d 100644
> --- a/arch/arm64/include/asm/kvm_emulate.h
> +++ b/arch/arm64/include/asm/kvm_emulate.h
> @@ -506,6 +506,15 @@ static inline unsigned long kvm_vcpu_get_mpidr_aff(struct kvm_vcpu *vcpu)
>  	return __vcpu_sys_reg(vcpu, MPIDR_EL1) & MPIDR_HWID_BITMASK;
>  }
>  
> +/*
> + * Check if vCPU MPIDR_EL1 has been reset. MPIDR_EL1.bit[31] is RES1
> + * and set during reset; a zero value indicates the vCPU is unreset.
> + */
> +static inline bool kvm_vcpu_mpidr_is_reset(struct kvm_vcpu *vcpu)
> +{
> +	return !!(__vcpu_sys_reg(vcpu, MPIDR_EL1) & MPIDR_RES1_BITMASK);
> +}
> +

This helper has only a single caller, so it would better be placed in
the function that makes use of it, specially considering that its a
one-liner.

>  static inline void kvm_vcpu_set_be(struct kvm_vcpu *vcpu)
>  {
>  	if (vcpu_mode_is_32bit(vcpu)) {
> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
> index 3732ee9eb0d4..fccfa97370df 100644
> --- a/arch/arm64/kvm/arm.c
> +++ b/arch/arm64/kvm/arm.c
> @@ -887,8 +887,18 @@ static void kvm_init_mpidr_data(struct kvm *kvm)
>  	data->mpidr_mask = mask;
>  
>  	kvm_for_each_vcpu(c, vcpu, kvm) {
> -		u64 aff = kvm_vcpu_get_mpidr_aff(vcpu);
> -		u16 index = kvm_mpidr_index(data, aff);
> +		u64 aff;
> +		u16 index;
> +
> +		/*
> +		 * Skip vCPUs that haven't been reset yet; their MPIDR_EL1 is
> +		 * zero.
> +		 */
> +		if (!kvm_vcpu_mpidr_is_reset(vcpu))
> +			continue;

But what about the initial loop that computes the significant bits
amongst the vcpus?

> +
> +		aff = kvm_vcpu_get_mpidr_aff(vcpu);
> +		index = kvm_mpidr_index(data, aff);

In all honesty, I think this is a userspace bug more than anything
else, and checking for random bits in MPDR_EL1 to verify whether the
value is plausible is gross.

Yhis isn't different from setting MPIDR_EL1 to the same value on all
vcpus, which we don't try to mitigate. Late setting of MPIDR_EL1 also
defeats the whole point of having a cache for the affinity to index
conversion, making SGIs pretty slow for late CPUs.

I really think that by not finalising your vcpus and start running the
guest, you have cornered yourself pretty badly, and we shouldn't try
to paper over it.

Thanks,

	M.

-- 
Jazz isn't dead. It just smells funny.