* [PATCH v3 1/2] KVM: arm64: Fix spurious warning for benign stage 2 teardown race
2026-09-01 17:28 [PATCH v3 0/2] KVM: arm64: Fix spurious warn, null ptr deref on S2 teardown race Lorenzo Stoakes (ARM)
@ 2026-09-01 17:28 ` Lorenzo Stoakes (ARM)
2026-09-01 17:29 ` [PATCH v3 2/2] KVM: arm64: nv: Fix null ptr deref on nested wp/unmap, " Lorenzo Stoakes (ARM)
1 sibling, 0 replies; 3+ messages in thread
From: Lorenzo Stoakes (ARM) @ 2026-09-01 17:28 UTC (permalink / raw)
To: Marc Zyngier, Oliver Upton, Joey Gouly, Steffen Eiden,
Suzuki K Poulose, Zenghui Yu, Catalin Marinas, Will Deacon,
Christoffer Dall, Fuad Tabba
Cc: Wei-Lin Chang, Yao Yuan, linux-arm-kernel, kvmarm, linux-kernel,
Lorenzo Stoakes (ARM), stable
kvmtool was used to establish an L1 guest with 8 CPUs and 8 GiB of RAM, an
L2 guest with 4 CPUs and 4 GiB of RAM and an L3 guest with 2 CPUs and 2 GiB
of RAM, all of which was then exited.
Under memory pressure in the L0 host warnings were observed due to
migration triggered by compaction:
WARNING: arch/arm64/kvm/mmu.c:336 at __unmap_stage2_range+0x64/0x80,
CPU#5: kcompactd0/66
Which was, in turn, triggered by an MMU notifier for the host invalidation:
mmu_notifier_invalidate_range_start()
-> ... -> kvm_mmu_notifier_invalidate_range_start()
-> kvm_mmu_unmap_gfn_range()
-> kvm_unmap_gfn_range()
-> kvm_nested_s2_unmap()
-> kvm_stage2_unmap_range()
-> __unmap_stage2_range()
-> stage2_apply_range()
<- -EINVAL, triggering a WARN_ON()
Racing with L0's teardown of stage 2 page tables:
exit_mm()
-> mmput()
-> __mmput()
-> exit_mmap()
-> mmu_notifier_release()
-> ... -> kvm_mmu_notifier_release()
-> kvm_flush_shadow_all()
-> kvm_arch_flush_shadow_all()
-> kvm_free_stage2_pgd()
-> [ acquire kvm->mmu_lock for write ]
-> mmu->pgt = NULL [ among other tasks ]
-> [ release kvm->mmu_lock for write ]
It turns out there is a benign race resulting in a spurious warning:
Thread A - notify: migration | Thread B - notify: release
-------------------------------|---------------------------------
< kvm->mmu_lock held > |
stage2_apply_range() |
get mmu->pgt, check !NULL |
... | kvm_arch_flush_shadow_all()
cond_resched_rwlock_write(); | < contend, sleep kvm->mmu_lock >
< drop kvm->mmu_lock > | < acquire kvm->mmu_lock>
| ...
| kvm_free_stage2_pgd()
| mmu->pgt = NULL
| < invalidate MMU >
| ...
| < release kvm->mmu_lock >
[ scheduled ] |
stage2_apply_range() |
< loop to next > |
get, mmu->pgt, check !NULL |
is NULL, return -EINVAL |
__unmap_stage2_range() |
WARN_ON(-EINVAL) <--- entirely spurious - the race was handled
correctly.
Fix the spurious warning by updating stage2_apply_range() to no longer
treat concurrent PGT teardown on lock release as an error - whether the
walker is tearing down page tables or doing something else this is a
legitimate reason to abort the operation without error.
This keeps the warning in place for all other circumstances.
In practice only __unmap_stage2_range() actually does anything with the
error so this only impacts that.
Fixes: ec14c272408a ("KVM: arm64: nv: Unmap/flush shadow stage 2 page tables")
Cc: stable@vger.kernel.org
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Reviewed-by: Marc Zyngier <maz@kernel.org>
Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
---
arch/arm64/kvm/mmu.c | 15 ++++++++++++---
1 file changed, 12 insertions(+), 3 deletions(-)
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 9ba86450fe4a..2d44cd6a5aed 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -59,27 +59,36 @@ static phys_addr_t stage2_range_addr_end(phys_addr_t addr, phys_addr_t end)
* long will also starve other vCPUs. We have to also make sure that the page
* tables are not freed while we released the lock.
*/
-static int stage2_apply_range(struct kvm_s2_mmu *mmu, phys_addr_t addr,
+static int stage2_apply_range(struct kvm_s2_mmu *mmu, phys_addr_t start,
phys_addr_t end,
int (*fn)(struct kvm_pgtable *, u64, u64),
bool resched)
{
struct kvm *kvm = kvm_s2_mmu_to_kvm(mmu);
+ bool lock_dropped = false;
+ phys_addr_t addr = start;
int ret;
u64 next;
do {
struct kvm_pgtable *pgt = mmu->pgt;
+ /*
+ * We may be raced on PGT teardown when we release the
+ * kvm->mmu_lock. That's fine as the PGT is legitimately no
+ * longer present.
+ */
if (!pgt)
- return -EINVAL;
+ return lock_dropped ? 0 : -EINVAL;
next = stage2_range_addr_end(addr, end);
ret = fn(pgt, addr, next - addr);
if (ret)
break;
- if (resched && next != end)
+ if (resched && next != end) {
cond_resched_rwlock_write(&kvm->mmu_lock);
+ lock_dropped = true;
+ }
} while (addr = next, addr != end);
return ret;
--
2.55.0
^ permalink raw reply related [flat|nested] 3+ messages in thread* [PATCH v3 2/2] KVM: arm64: nv: Fix null ptr deref on nested wp/unmap, teardown race
2026-09-01 17:28 [PATCH v3 0/2] KVM: arm64: Fix spurious warn, null ptr deref on S2 teardown race Lorenzo Stoakes (ARM)
2026-09-01 17:28 ` [PATCH v3 1/2] KVM: arm64: Fix spurious warning for benign stage 2 " Lorenzo Stoakes (ARM)
@ 2026-09-01 17:29 ` Lorenzo Stoakes (ARM)
1 sibling, 0 replies; 3+ messages in thread
From: Lorenzo Stoakes (ARM) @ 2026-09-01 17:29 UTC (permalink / raw)
To: Marc Zyngier, Oliver Upton, Joey Gouly, Steffen Eiden,
Suzuki K Poulose, Zenghui Yu, Catalin Marinas, Will Deacon,
Christoffer Dall, Fuad Tabba
Cc: Wei-Lin Chang, Yao Yuan, linux-arm-kernel, kvmarm, linux-kernel,
Lorenzo Stoakes (ARM), stable
Commit 7270cc9157f4 ("KVM: arm64: nv: Handle VNCR_EL2 invalidation from MMU
notifiers") introduced VNCR_EL2 invalidation in both kvm_nested_s2_unmap()
and kvm_nested_s2_wp().
However at the point of this being performed concurrent stage 2 teardown of
a nested guest can cause kvm->arch.mmu.pgt to be set to NULL.
This happens in kvm_flush_shadow_all() -> kvm_arch_flush_shadow_all() ->
kvm_free_stage2_pgd() and is performed under the kvm->mmu_lock.
Commit ec14c272408a ("KVM: arm64: nv: Unmap/flush shadow stage 2 page
tables") introduced the teardown of the entire nested MMU range, which then
invokes stage2_apply_range() with resched=true:
mmu_notifier_invalidate_range_start()
-> ... -> kvm_mmu_notifier_invalidate_range_start()
-> kvm_mmu_unmap_gfn_range()
-> kvm_unmap_gfn_range()
-> kvm_nested_s2_unmap()
-> kvm_stage2_unmap_range()
-> __unmap_stage2_range()
-> stage2_apply_range()
This means that stage2_apply_range() can drop the kvm->mmu_lock and thus
concurrent progress can be made in lockstep with
kvm_arch_flush_shadow_all().
If kvm_arch_flush_shadow_all() advances ahead of stage2_apply_range() and
completes its operation it guarantees a NULL pointer deref.
Since kvm_free_stage2_pgd() is performed under the kvm->mmu_lock this will
either be observed NULL or not and serialised against
kvm_free_stage2_pgd().
Resolve the issue by abstracting the invalidation to a new function,
kvm_invalidate_vncr_ipa_all(), and check that the pgt is non-NULL before
dereferencing it.
Fixes: 7270cc9157f4 ("KVM: arm64: nv: Handle VNCR_EL2 invalidation from MMU notifiers")
Cc: stable@vger.kernel.org
Reviewed-by: Marc Zyngier <maz@kernel.org>
Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
---
arch/arm64/kvm/nested.c | 15 +++++++++++++--
1 file changed, 13 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index 17123f0b6dab..f69722e1592a 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -1260,6 +1260,17 @@ void kvm_handle_s1e2_tlbi(struct kvm_vcpu *vcpu, u32 inst, u64 val)
invalidate_vncr_va(vcpu->kvm, &scope);
}
+static void kvm_invalidate_vncr_ipa_all(struct kvm *kvm)
+{
+ struct kvm_pgtable *pgt = kvm->arch.mmu.pgt;
+
+ lockdep_assert_held_write(&kvm->mmu_lock);
+
+ /* if the mmu lock was dropped, pgt teardown may have raced. */
+ if (pgt)
+ kvm_invalidate_vncr_ipa(kvm, 0, BIT(pgt->ia_bits));
+}
+
void kvm_nested_s2_wp(struct kvm *kvm)
{
int i;
@@ -1276,7 +1287,7 @@ void kvm_nested_s2_wp(struct kvm *kvm)
kvm_stage2_wp_range(mmu, 0, kvm_phys_size(mmu));
}
- kvm_invalidate_vncr_ipa(kvm, 0, BIT(kvm->arch.mmu.pgt->ia_bits));
+ kvm_invalidate_vncr_ipa_all(kvm);
}
void kvm_nested_s2_unmap(struct kvm *kvm, bool may_block)
@@ -1295,7 +1306,7 @@ void kvm_nested_s2_unmap(struct kvm *kvm, bool may_block)
kvm_stage2_unmap_range(mmu, 0, kvm_phys_size(mmu), may_block);
}
- kvm_invalidate_vncr_ipa(kvm, 0, BIT(kvm->arch.mmu.pgt->ia_bits));
+ kvm_invalidate_vncr_ipa_all(kvm);
}
void kvm_nested_s2_flush(struct kvm *kvm)
--
2.55.0
^ permalink raw reply related [flat|nested] 3+ messages in thread