From: "Lorenzo Stoakes (ARM)" <ljs@kernel.org>
To: Marc Zyngier <maz@kernel.org>, Oliver Upton <oupton@kernel.org>,
Fuad Tabba <tabba@google.com>, Joey Gouly <joey.gouly@arm.com>,
Steffen Eiden <seiden@linux.ibm.com>,
Suzuki K Poulose <suzuki.poulose@arm.com>,
Zenghui Yu <yuzenghui@huawei.com>,
Catalin Marinas <catalin.marinas@arm.com>,
Will Deacon <will@kernel.org>,
Jintack Lim <jintack.lim@linaro.org>,
Christoffer Dall <christoffer.dall@linaro.org>,
Christoffer Dall <christoffer.dall@arm.com>
Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev,
linux-kernel@vger.kernel.org, stable@vger.kernel.org
Subject: Re: [PATCH 1/2] KVM: arm64: Fix spurious warning for benign stage 2 teardown race
Date: Wed, 12 Aug 2026 14:46:25 +0100 [thread overview]
Message-ID: <anx42VidDlhWKBYt@gremlin> (raw)
In-Reply-To: <20260812-kvm-arm-nested-virt-fix-v1-1-4ad883f1b6a5@kernel.org>
The splats:
------------[ cut here ]------------
WARNING: arch/arm64/kvm/mmu.c:336 at __unmap_stage2_range+0x64/0x80, CPU#5: kcompactd0/66
...
Tainted: [S]=CPU_OUT_OF_SPEC
Hardware name: Apple Inc. Mac14,7/J493, BIOS 2026.04 04/01/2026
pstate: 61401009 (nZCv daif +PAN -UAO -TCO +DIT +SSBS BTYPE=--)
pc : __unmap_stage2_range (arch/arm64/kvm/mmu.c:335 (discriminator 3))
lr : __unmap_stage2_range (arch/arm64/kvm/mmu.c:336 (discriminator 5))
sp : ffff8000856e7450
x29: ffff8000856e7450 x28: ffff0000404b2000 x27: 0000000000000030
x26: ffffffffffffffe0 x25: 0000000000000001 x24: ffff00000eaa5920
x23: ffff0000404b2000 x22: 0000000000000000 x21: 0000000000000040
x20: 0000000000000001 x19: 0000000000000001 x18: 0000000000000006
x17: 0000000000000000 x16: 0000000000000002 x15: ffff800083adbed0
x14: ffff800083adbed0 x13: 0000000000000008 x12: 0000b00000000000
x11: 0000000000000001 x10: 0000006000000000 x9 : ffff8000817334fc
x8 : 0000000000200000 x7 : ffff0000c59ea000 x6 : 0000000000000800
x5 : 0000000038000000 x4 : 0000000000000000 x3 : ffff0000404b2000
x2 : 0000000000000001 x1 : 0000000038000000 x0 : 00000000ffffffea
Call trace:
__unmap_stage2_range (arch/arm64/kvm/mmu.c:335 (discriminator 3)) (P)
kvm_stage2_unmap_range (arch/arm64/kvm/mmu.c:346)
kvm_nested_s2_unmap (arch/arm64/kvm/nested.c:1168 (discriminator 14))
kvm_unmap_gfn_range (arch/arm64/kvm/mmu.c:2416)
kvm_mmu_notifier_invalidate_range_start (arch/arm64/kvm/../../../virt/kvm/kvm_main.c:718 (discriminator 32768) arch/arm64/kvm/../../../virt/kvm/kvm_main.c:633 (discriminator 32768) arch/arm64/kvm/../../../virt/kvm/kvm_main.c:765 (discriminator 32768))
mn_hlist_invalidate_range_start (mm/mmu_notifier.c:525)
__mmu_notifier_invalidate_range_start (mm/mmu_notifier.c:580)
try_to_migrate_one (./include/linux/mmu_notifier.h:478 ./include/linux/mmu_notifier.h:471 mm/rmap.c:2459)
rmap_walk_anon (mm/rmap.c:3001)
rmap_walk (mm/rmap.c:3106 mm/rmap.c:3101)
try_to_migrate (mm/rmap.c:2774)
migrate_folio_unmap (mm/migrate.c:1330 (discriminator 3))
migrate_pages_batch (mm/migrate.c:1909)
migrate_pages_sync (mm/migrate.c:2026)
migrate_pages (mm/migrate.c:2135)
compact_zone (mm/compaction.c:2663)
compact_node (mm/compaction.c:2932)
kcompactd (mm/compaction.c:3230)
kthread (kernel/kthread.c:436)
ret_from_fork (arch/arm64/kernel/entry.S:858)
---[ end trace 0000000000000000 ]---
------------[ cut here ]------------
WARNING: arch/arm64/kvm/mmu.c:336 at __unmap_stage2_range+0x64/0x80, CPU#5: kcompactd0/66
...
Tainted: [S]=CPU_OUT_OF_SPEC, [W]=WARN
Hardware name: Apple Inc. Mac14,7/J493, BIOS 2026.04 04/01/2026
pstate: 61401009 (nZCv daif +PAN -UAO -TCO +DIT +SSBS BTYPE=--)
pc : __unmap_stage2_range (arch/arm64/kvm/mmu.c:335 (discriminator 3))
lr : __unmap_stage2_range (arch/arm64/kvm/mmu.c:336 (discriminator 5))
sp : ffff8000856e7450
x29: ffff8000856e7450 x28: ffff0000404b2000 x27: 0000000000000030
x26: ffffffffffffffe0 x25: 0000000000000001 x24: ffff00000eaa5920
x23: ffff0000404b2000 x22: 0000000000000001 x21: 0000000000000040
x20: 0000000000000001 x19: 0000000000000001 x18: 0000000000000006
x17: 0000000000000000 x16: 0000000000000002 x15: ffff800083adbed0
x14: ffff800083adbed0 x13: 0000000000000008 x12: 0000b00000000000
x11: 0000000000000001 x10: 0000006000000000 x9 : ffff8000817334fc
x8 : 0000000000200000 x7 : ffff0000c59ea078 x6 : 0000000000000800
x5 : 0000000002000000 x4 : 0000000000000000 x3 : ffff0000404b2000
x2 : 0000000000000001 x1 : 0000000002000000 x0 : 00000000ffffffea
Call trace:
__unmap_stage2_range (arch/arm64/kvm/mmu.c:335 (discriminator 3)) (P)
kvm_stage2_unmap_range (arch/arm64/kvm/mmu.c:346)
kvm_nested_s2_unmap (arch/arm64/kvm/nested.c:1168 (discriminator 14))
kvm_unmap_gfn_range (arch/arm64/kvm/mmu.c:2416)
kvm_mmu_notifier_invalidate_range_start (arch/arm64/kvm/../../../virt/kvm/kvm_main.c:718 (discriminator 32768) arch/arm64/kvm/../../../virt/kvm/kvm_main.c:633 (discriminator 32768) arch/arm64/kvm/../../../virt/kvm/kvm_main.c:765 (discriminator 32768))
mn_hlist_invalidate_range_start (mm/mmu_notifier.c:525)
__mmu_notifier_invalidate_range_start (mm/mmu_notifier.c:580)
try_to_migrate_one (./include/linux/mmu_notifier.h:478 ./include/linux/mmu_notifier.h:471 mm/rmap.c:2459)
rmap_walk_anon (mm/rmap.c:3001)
rmap_walk (mm/rmap.c:3106 mm/rmap.c:3101)
try_to_migrate (mm/rmap.c:2774)
migrate_folio_unmap (mm/migrate.c:1330 (discriminator 3))
migrate_pages_batch (mm/migrate.c:1909)
migrate_pages_sync (mm/migrate.c:2026)
migrate_pages (mm/migrate.c:2135)
compact_zone (mm/compaction.c:2663)
compact_node (mm/compaction.c:2932)
kcompactd (mm/compaction.c:3230)
kthread (kernel/kthread.c:436)
ret_from_fork (arch/arm64/kernel/entry.S:858)
---[ end trace 0000000000000000 ]---
------------[ cut here ]------------
WARNING: arch/arm64/kvm/mmu.c:336 at __unmap_stage2_range+0x64/0x80, CPU#5: kcompactd0/66
...
Tainted: [S]=CPU_OUT_OF_SPEC, [W]=WARN
Hardware name: Apple Inc. Mac14,7/J493, BIOS 2026.04 04/01/2026
pstate: 61401009 (nZCv daif +PAN -UAO -TCO +DIT +SSBS BTYPE=--)
pc : __unmap_stage2_range (arch/arm64/kvm/mmu.c:335 (discriminator 3))
lr : __unmap_stage2_range (arch/arm64/kvm/mmu.c:336 (discriminator 5))
sp : ffff8000856e7450
x29: ffff8000856e7450 x28: ffff0000404b2000 x27: 0000000000000030
x26: ffffffffffffffe0 x25: 0000000000000001 x24: ffff00000eaa5920
x23: ffff0000404b2000 x22: 0000000000000002 x21: 0000000000000040
x20: 0000000000000001 x19: 0000000000000001 x18: 0000000000000006
x17: 0000000000000000 x16: 0000000000000002 x15: ffff800083adbed0
x14: ffff800083adbed0 x13: 0000000000000008 x12: 0000b00000000000
x11: 0000000000000001 x10: 0000006000000000 x9 : ffff8000817334fc
x8 : 0000000000200000 x7 : ffff0000c59ea0f0 x6 : 0000000000000800
x5 : 0000000002000000 x4 : 0000000000000000 x3 : ffff0000404b2000
x2 : 0000000000000001 x1 : 0000000002000000 x0 : 00000000ffffffea
Call trace:
__unmap_stage2_range (arch/arm64/kvm/mmu.c:335 (discriminator 3)) (P)
kvm_stage2_unmap_range (arch/arm64/kvm/mmu.c:346)
kvm_nested_s2_unmap (arch/arm64/kvm/nested.c:1168 (discriminator 14))
kvm_unmap_gfn_range (arch/arm64/kvm/mmu.c:2416)
kvm_mmu_notifier_invalidate_range_start (arch/arm64/kvm/../../../virt/kvm/kvm_main.c:718 (discriminator 32768) arch/arm64/kvm/../../../virt/kvm/kvm_main.c:633 (discriminator 32768) arch/arm64/kvm/../../../virt/kvm/kvm_main.c:765 (discriminator 32768))
mn_hlist_invalidate_range_start (mm/mmu_notifier.c:525)
__mmu_notifier_invalidate_range_start (mm/mmu_notifier.c:580)
try_to_migrate_one (./include/linux/mmu_notifier.h:478 ./include/linux/mmu_notifier.h:471 mm/rmap.c:2459)
rmap_walk_anon (mm/rmap.c:3001)
rmap_walk (mm/rmap.c:3106 mm/rmap.c:3101)
try_to_migrate (mm/rmap.c:2774)
migrate_folio_unmap (mm/migrate.c:1330 (discriminator 3))
migrate_pages_batch (mm/migrate.c:1909)
migrate_pages_sync (mm/migrate.c:2026)
migrate_pages (mm/migrate.c:2135)
compact_zone (mm/compaction.c:2663)
compact_node (mm/compaction.c:2932)
kcompactd (mm/compaction.c:3230)
kthread (kernel/kthread.c:436)
ret_from_fork (arch/arm64/kernel/entry.S:858)
---[ end trace 0000000000000000 ]---
--
Also I wrote something extra but dropped from the already-far-too-long commit
msg but the registers match the diagnosis, basically:
--
The observed assembly from the splats is consistent with this analysis:
Examining the assembly for stage2_apply_range() loop and pgt check,
conveniently x1 contains the current IPA being examined (addr) and is not
clobbered on return:
mov x1, x19 addr = next
ldr x0, [x26, #16] struct kvm_pgtable *pgt = mmu->pgt;
cbnz x0, ... if (!pgt)...
ldp x19, x20, [sp, #16] }
mov w0, #0xffffffea } return -EINVAL
ret }
Splat 1 has x1 of 0x38000000 which tells us the splat occurred 27 * 32 MiB
or 864 MiB into the nested guest physical address space (27 iterations).
This is consistent with an an interleaving stage 2 tear down occurring
part-way through this operation.
There being 2 other splats is consistent with there being a total of 3
nested mmus, as expected, and that the mmu is invalidated before the
warning can occur.
The remaining 2 have x1 equal to 0x2000000 or exactly 32 MiB, which is what
you would expect with the stage2_apply_range() and
kvm_arch_flush_shadow_all() running in lockstep across mmus:
kvm_unmap_gfn_range() | kvm_arch_flush_shadow_all()
---------------------------------|----------------------------
< kvm->mmu_lock held > |
stage2_apply_range() |
.. 26 successful iterations .. | < contend kvm->mmu_lock >
< releases kvm->mmu lock > | < acquires kvm->mmu_lock >
| mmu->pgt = NULL
| < releases kvm->mmu_lock >
Then 3 occurrences of:
< reacquires kvm->mmu_lock > |
return to __unmap_stage2_range() |
WARN_ON(-EINVAL); |
mmu = &kvm->arch.nested_mmus[1]; |
stage2_apply_range() |
.. 1 successful iteration .. |
< releases kvm->mmu lock > | < contend kvm->mmu_lock >
| < acquires kvm->mmu_lock >
| mmu->pgt = NULL
| < releases kvm->mmu_lock >
One single iteration sets addr = next = 0x2000000 which is reflected in the
splats.
Finally, in kvm_nested_s2_unmap() the disassembly indicates that w22
maintains the i counter, which happens not to be clobbered over the
operation either:
mov w22, #0x0 for (i = 0;
add w22, w22, #0x1 ; i++)
sxtw x2, w22 }
cmp x2, x1 } i < nested_mmus_size
The splats have w22 equal to 0, 1 and 2 respectively which further
corroborates this analysis.
--
Cheers, Lorenzo
next prev parent reply other threads:[~2026-08-12 13:46 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-08-12 13:31 [PATCH 0/2] KVM: arm64: Fix spurious warn, null ptr deref on S2 teardown race Lorenzo Stoakes (ARM)
2026-08-12 13:31 ` [PATCH 1/2] KVM: arm64: Fix spurious warning for benign stage 2 " Lorenzo Stoakes (ARM)
2026-08-12 13:46 ` Lorenzo Stoakes (ARM) [this message]
2026-08-12 13:54 ` sashiko-bot
2026-08-12 13:31 ` [PATCH 2/2] KVM: arm64: nv: Fix null ptr deref in kvm_nested_s2_unmap() on S2 teardown Lorenzo Stoakes (ARM)
2026-08-12 14:01 ` sashiko-bot
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=anx42VidDlhWKBYt@gremlin \
--to=ljs@kernel.org \
--cc=catalin.marinas@arm.com \
--cc=christoffer.dall@arm.com \
--cc=christoffer.dall@linaro.org \
--cc=jintack.lim@linaro.org \
--cc=joey.gouly@arm.com \
--cc=kvmarm@lists.linux.dev \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=maz@kernel.org \
--cc=oupton@kernel.org \
--cc=seiden@linux.ibm.com \
--cc=stable@vger.kernel.org \
--cc=suzuki.poulose@arm.com \
--cc=tabba@google.com \
--cc=will@kernel.org \
--cc=yuzenghui@huawei.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.