* [PATCH 2/2] KVM: arm64: nv: Update the shadow MMU's VMID for AT emulation
2026-11-16 6:00 [PATCH 0/2] KVM: arm64: nv: fix a couple bugs in AT emulation Dan Carpenter
@ 2026-11-16 6:00 ` Dan Carpenter
2026-11-16 6:00 ` [PATCH 1/2] KVM: arm64: nv: Fix AT emulation for TCR2_EL1 Dan Carpenter
1 sibling, 0 replies; 4+ messages in thread
From: Dan Carpenter @ 2026-11-16 6:00 UTC (permalink / raw)
To: Marc Zyngier
Cc: Oliver Upton, Fuad Tabba, Joey Gouly, Steffen Eiden,
Suzuki K Poulose, Zenghui Yu, Catalin Marinas, Will Deacon,
Mark Rutland, Jintack Lim, linux-arm-kernel, kvmarm, linux-kernel,
Karthik Bhargavan, Clément Blaudeau
There is a missing kvm_arm_vmid_update() before the __load_stage2()
in __kvm_at_s1e01_fast(). The bug affects nested virtualization on
arm64.
In KVM a "stage 2" refers to the VM's page table mappings. The
__kvm_at_s1e01_fast() function emulates the AT instruction, which
translates a virtual address the way the MMU would and returns the
result.
Each stage 2 has a VMID, which the hardware uses to tag TLB and
walk-cache entries, so that one VM never uses another's entries. There
are only 2^8 or 2^16 VMIDs, which is quite limited, so vmid.c hands
them out by generation. When they run out, a rollover starts a new
generation and flushes the TLBs: the VMIDs active on a CPU at that
moment are kept, and every other VMID can be given to another VM.
The kvm_arm_vmid_update() function checks the VMID's generation, takes
a new VMID if it is from an old generation, and marks it as active on
this CPU. Without this, the shadow MMU's VMID could be an old one that
now belongs to another VM, or a rollover on another CPU could hand it
to another VM while we are using it. Either way the AT fills TLB entries
tagged with another VM's VMID.
To fix this, call kvm_arm_vmid_update() to update the VMID before loading
the shadow stage 2. That marks it as active on the CPU. Then at the end
of the function, undo that. Restore the original VMID as active while
checking for and handling rollover.
Fixes: 477e89cabb14 ("KVM: arm64: nv: Add basic emulation of AT S1E{0,1}{R,W}")
Cc: stable@vger.kernel.org
Signed-off-by: Dan Carpenter <dan@pereq.eu>
---
arch/arm64/kvm/at.c | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/kvm/at.c b/arch/arm64/kvm/at.c
index 06765881a4c8..34c9d10188e0 100644
--- a/arch/arm64/kvm/at.c
+++ b/arch/arm64/kvm/at.c
@@ -1455,6 +1455,7 @@ static u64 __kvm_at_s1e01_fast(struct kvm_vcpu *vcpu, u32 op, u64 vaddr)
}
}
write_sysreg_el1(vcpu_read_sys_reg(vcpu, SCTLR_EL1), SYS_SCTLR);
+ kvm_arm_vmid_update(&mmu->vmid);
__load_stage2(mmu);
skip_mmu_switch:
@@ -1496,8 +1497,11 @@ static u64 __kvm_at_s1e01_fast(struct kvm_vcpu *vcpu, u32 op, u64 vaddr)
write_sysreg_hcr(HCR_HOST_VHE_FLAGS);
- if (mmu_cs)
+ if (mmu_cs) {
+ kvm_arm_vmid_update(&vcpu->arch.hw_mmu->vmid);
+ config.vttbr = kvm_get_vttbr(vcpu->arch.hw_mmu);
__mmu_config_restore(&config);
+ }
return par;
}
--
2.53.0
^ permalink raw reply related [flat|nested] 4+ messages in thread
* [PATCH 0/2] KVM: arm64: nv: fix a couple bugs in AT emulation
@ 2026-11-16 6:00 Dan Carpenter
2026-11-16 6:00 ` [PATCH 2/2] KVM: arm64: nv: Update the shadow MMU's VMID for " Dan Carpenter
2026-11-16 6:00 ` [PATCH 1/2] KVM: arm64: nv: Fix AT emulation for TCR2_EL1 Dan Carpenter
0 siblings, 2 replies; 4+ messages in thread
From: Dan Carpenter @ 2026-11-16 6:00 UTC (permalink / raw)
To: Marc Zyngier
Cc: Catalin Marinas, Fuad Tabba, Jintack Lim, Joey Gouly, kvmarm,
linux-arm-kernel, linux-kernel, Mark Rutland, Oliver Upton,
Steffen Eiden, Suzuki K Poulose, Will Deacon, Zenghui Yu,
Karthik Bhargavan, Clément Blaudeau
These are two bugs in __kvm_at_s1e01_fast() which affect nested
virtualization on arm64.
They were found as part of our Formal Analysis work at Pereq.
I have tried my best to get things right and I have tested the results
but please understand that I am a newbie in this area and review
accordingly.
Dan Carpenter (2):
KVM: arm64: nv: Fix AT emulation for TCR2_EL1
KVM: arm64: nv: Update the shadow MMU's VMID for AT emulation
arch/arm64/kvm/at.c | 20 ++++++++++++++++++--
1 file changed, 18 insertions(+), 2 deletions(-)
--
2.53.0
^ permalink raw reply [flat|nested] 4+ messages in thread
* [PATCH 1/2] KVM: arm64: nv: Fix AT emulation for TCR2_EL1
2026-11-16 6:00 [PATCH 0/2] KVM: arm64: nv: fix a couple bugs in AT emulation Dan Carpenter
2026-11-16 6:00 ` [PATCH 2/2] KVM: arm64: nv: Update the shadow MMU's VMID for " Dan Carpenter
@ 2026-11-16 6:00 ` Dan Carpenter
2026-11-16 16:01 ` Marc Zyngier
1 sibling, 1 reply; 4+ messages in thread
From: Dan Carpenter @ 2026-11-16 6:00 UTC (permalink / raw)
To: Marc Zyngier
Cc: Oliver Upton, Fuad Tabba, Joey Gouly, Steffen Eiden,
Suzuki K Poulose, Zenghui Yu, Catalin Marinas, Will Deacon,
Mark Rutland, Jintack Lim, linux-arm-kernel, kvmarm, linux-kernel,
Karthik Bhargavan, Clément Blaudeau
Arm provides an AT instruction that translates a virtual address the way
the MMU would and returns the result. There is a bug in the way this
instruction is emulated for nested virtualization because of how TCR2_EL1
is handled. The bug is fairly minor.
The TCR2_EL1 register has bits which turn on permission indirection
(PIE) and permission overlays (POE). When these bits are set, the AT
instruction reads the permissions stored in PIR_EL1 and POR_EL1. If
you have the right features and permissions then in the return value from
the AT instruction we set one bit to either 1 or 0 depending on whether
the access would be allowed or whether it would fault.
However, the bug is that just because the host has TCR2 enabled, it
doesn't mean the VM has it. KVM doesn't load these registers for a VM
which doesn't have the features, instead the AT runs with the host's
HCRX_EL2, which has TCR2En set. So the AT uses the TCR2_EL1, PIR_EL1 and
POR_EL1 values another VM left on the CPU, or possibly the VMM's POR_EL0.
Likewise, for a VM which has TCR2 but not PIE or POE, the L2's TCR2_EL1
is loaded unmasked.
The impact of this bug is that we leak a single bit of data to the L1,
but in real life it's hard to imagine how it is useful to an attacker.
Fix this by writing zero to TCR2_EL1 when the VM doesn't have TCR2, and
by clearing the PIE and POE bits when the VM doesn't have those
features. Also it should be effective_tcr2(vcpu, TR_EL10) to read the
TCR2 for the L1 instead of vcpu_read_sys_reg(vcpu, TCR2_EL1).
Fixes: 477e89cabb14 ("KVM: arm64: nv: Add basic emulation of AT S1E{0,1}{R,W}")
Fixes: 23e7a34c8397 ("KVM: arm64: Add AT fast-path support for S1PIE")
Cc: stable@vger.kernel.org
Signed-off-by: Dan Carpenter <dan@pereq.eu>
---
arch/arm64/kvm/at.c | 14 +++++++++++++-
1 file changed, 13 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/kvm/at.c b/arch/arm64/kvm/at.c
index 30d8f062b2c2..06765881a4c8 100644
--- a/arch/arm64/kvm/at.c
+++ b/arch/arm64/kvm/at.c
@@ -1438,7 +1438,13 @@ static u64 __kvm_at_s1e01_fast(struct kvm_vcpu *vcpu, u32 op, u64 vaddr)
write_sysreg_el1(vcpu_read_sys_reg(vcpu, TCR_EL1), SYS_TCR);
write_sysreg_el1(vcpu_read_sys_reg(vcpu, MAIR_EL1), SYS_MAIR);
if (kvm_has_tcr2(vcpu->kvm)) {
- write_sysreg_el1(vcpu_read_sys_reg(vcpu, TCR2_EL1), SYS_TCR2);
+ u64 tcr2 = effective_tcr2(vcpu, TR_EL10);
+
+ if (!kvm_has_s1pie(vcpu->kvm))
+ tcr2 &= ~TCR2_EL1_PIE;
+ if (!kvm_has_s1poe(vcpu->kvm))
+ tcr2 &= ~(TCR2_EL1_POE | TCR2_EL1_E0POE);
+ write_sysreg_el1(tcr2, SYS_TCR2);
if (kvm_has_s1pie(vcpu->kvm)) {
write_sysreg_el1(vcpu_read_sys_reg(vcpu, PIR_EL1), SYS_PIR);
write_sysreg_el1(vcpu_read_sys_reg(vcpu, PIRE0_EL1), SYS_PIRE0);
@@ -1452,6 +1458,9 @@ static u64 __kvm_at_s1e01_fast(struct kvm_vcpu *vcpu, u32 op, u64 vaddr)
__load_stage2(mmu);
skip_mmu_switch:
+ if (cpus_have_final_cap(ARM64_HAS_TCR2) && !kvm_has_tcr2(vcpu->kvm))
+ write_sysreg_el1(0, SYS_TCR2);
+
/* Temporarily switch back to guest context */
write_sysreg_hcr(vcpu->arch.hcr_el2);
isb();
@@ -1554,6 +1563,9 @@ int __kvm_at_s1e2(struct kvm_vcpu *vcpu, u32 op, u64 vaddr)
if (!vcpu_el2_e2h_is_set(vcpu))
val |= HCR_EL2_NV | HCR_EL2_NV1;
+ if (cpus_have_final_cap(ARM64_HAS_TCR2) && !kvm_has_tcr2(vcpu->kvm))
+ write_sysreg_el1(0, SYS_TCR2);
+
write_sysreg_hcr(val);
isb();
--
2.53.0
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH 1/2] KVM: arm64: nv: Fix AT emulation for TCR2_EL1
2026-11-16 6:00 ` [PATCH 1/2] KVM: arm64: nv: Fix AT emulation for TCR2_EL1 Dan Carpenter
@ 2026-11-16 16:01 ` Marc Zyngier
0 siblings, 0 replies; 4+ messages in thread
From: Marc Zyngier @ 2026-11-16 16:01 UTC (permalink / raw)
To: Dan Carpenter
Cc: Oliver Upton, Fuad Tabba, Joey Gouly, Steffen Eiden,
Suzuki K Poulose, Zenghui Yu, Catalin Marinas, Will Deacon,
Mark Rutland, Jintack Lim, linux-arm-kernel, kvmarm, linux-kernel,
Karthik Bhargavan, Clément Blaudeau
On Mon, 16 Nov 2026 06:00:52 +0000,
Dan Carpenter <dan@pereq.eu> wrote:
>
> Arm provides an AT instruction that translates a virtual address the way
> the MMU would and returns the result. There is a bug in the way this
> instruction is emulated for nested virtualization because of how TCR2_EL1
> is handled. The bug is fairly minor.
>
> The TCR2_EL1 register has bits which turn on permission indirection
> (PIE) and permission overlays (POE). When these bits are set, the AT
> instruction reads the permissions stored in PIR_EL1 and POR_EL1. If
> you have the right features and permissions then in the return value from
> the AT instruction we set one bit to either 1 or 0 depending on whether
> the access would be allowed or whether it would fault.
>
> However, the bug is that just because the host has TCR2 enabled, it
> doesn't mean the VM has it. KVM doesn't load these registers for a VM
> which doesn't have the features, instead the AT runs with the host's
> HCRX_EL2, which has TCR2En set. So the AT uses the TCR2_EL1, PIR_EL1 and
> POR_EL1 values another VM left on the CPU, or possibly the VMM's POR_EL0.
> Likewise, for a VM which has TCR2 but not PIE or POE, the L2's TCR2_EL1
> is loaded unmasked.
>
> The impact of this bug is that we leak a single bit of data to the L1,
> but in real life it's hard to imagine how it is useful to an attacker.
>
> Fix this by writing zero to TCR2_EL1 when the VM doesn't have TCR2, and
> by clearing the PIE and POE bits when the VM doesn't have those
> features. Also it should be effective_tcr2(vcpu, TR_EL10) to read the
> TCR2 for the L1 instead of vcpu_read_sys_reg(vcpu, TCR2_EL1).
>
> Fixes: 477e89cabb14 ("KVM: arm64: nv: Add basic emulation of AT S1E{0,1}{R,W}")
> Fixes: 23e7a34c8397 ("KVM: arm64: Add AT fast-path support for S1PIE")
> Cc: stable@vger.kernel.org
> Signed-off-by: Dan Carpenter <dan@pereq.eu>
> ---
> arch/arm64/kvm/at.c | 14 +++++++++++++-
> 1 file changed, 13 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm64/kvm/at.c b/arch/arm64/kvm/at.c
> index 30d8f062b2c2..06765881a4c8 100644
> --- a/arch/arm64/kvm/at.c
> +++ b/arch/arm64/kvm/at.c
> @@ -1438,7 +1438,13 @@ static u64 __kvm_at_s1e01_fast(struct kvm_vcpu *vcpu, u32 op, u64 vaddr)
> write_sysreg_el1(vcpu_read_sys_reg(vcpu, TCR_EL1), SYS_TCR);
> write_sysreg_el1(vcpu_read_sys_reg(vcpu, MAIR_EL1), SYS_MAIR);
> if (kvm_has_tcr2(vcpu->kvm)) {
> - write_sysreg_el1(vcpu_read_sys_reg(vcpu, TCR2_EL1), SYS_TCR2);
> + u64 tcr2 = effective_tcr2(vcpu, TR_EL10);
> +
> + if (!kvm_has_s1pie(vcpu->kvm))
> + tcr2 &= ~TCR2_EL1_PIE;
> + if (!kvm_has_s1poe(vcpu->kvm))
> + tcr2 &= ~(TCR2_EL1_POE | TCR2_EL1_E0POE);
> + write_sysreg_el1(tcr2, SYS_TCR2);
The sanitising should really be moved into the effective_tcr2()
helper, as this is its purpose. Something like the below, which is
obviously untested... Ultimately, TCR2_EL1 should be moved to the
automatic sanitisation scheme to inherit the expected RES0 behaviours.
> if (kvm_has_s1pie(vcpu->kvm)) {
> write_sysreg_el1(vcpu_read_sys_reg(vcpu, PIR_EL1), SYS_PIR);
> write_sysreg_el1(vcpu_read_sys_reg(vcpu, PIRE0_EL1), SYS_PIRE0);
> @@ -1452,6 +1458,9 @@ static u64 __kvm_at_s1e01_fast(struct kvm_vcpu *vcpu, u32 op, u64 vaddr)
> __load_stage2(mmu);
>
> skip_mmu_switch:
> + if (cpus_have_final_cap(ARM64_HAS_TCR2) && !kvm_has_tcr2(vcpu->kvm))
> + write_sysreg_el1(0, SYS_TCR2);
> +
> /* Temporarily switch back to guest context */
> write_sysreg_hcr(vcpu->arch.hcr_el2);
> isb();
> @@ -1554,6 +1563,9 @@ int __kvm_at_s1e2(struct kvm_vcpu *vcpu, u32 op, u64 vaddr)
> if (!vcpu_el2_e2h_is_set(vcpu))
> val |= HCR_EL2_NV | HCR_EL2_NV1;
>
> + if (cpus_have_final_cap(ARM64_HAS_TCR2) && !kvm_has_tcr2(vcpu->kvm))
> + write_sysreg_el1(0, SYS_TCR2);
> +
> write_sysreg_hcr(val);
> isb();
>
This one needs fixing, as sashiko reported, but the reason why the
proposed fix works at all is pretty fragile:
- the reset value is 0
- we enforce RES0 behaviours on all interesting fields
- TCR2_ELx are trapped and result in an UNDEF
So this deserves a comment, along the lines of what I have written in
the hack below.
Thanks,
M.
diff --git a/arch/arm64/kvm/at.c b/arch/arm64/kvm/at.c
index 30d8f062b2c29..1ff452277f27e 100644
--- a/arch/arm64/kvm/at.c
+++ b/arch/arm64/kvm/at.c
@@ -91,11 +91,16 @@ static enum trans_regime compute_translation_regime(struct kvm_vcpu *vcpu, u32 o
static u64 effective_tcr2(struct kvm_vcpu *vcpu, enum trans_regime regime)
{
if (regime == TR_EL10) {
+ u64 val;
+
if (vcpu_has_nv(vcpu) &&
!(__vcpu_sys_reg(vcpu, HCRX_EL2) & HCRX_EL2_TCR2En))
return 0;
- return vcpu_read_sys_reg(vcpu, TCR2_EL1);
+ val = vcpu_read_sys_reg(vcpu, TCR2_EL1);
+ val &= ~(kvm_has_s1poe(vcpu->kvm) ? (TCR2_EL1_POE | TCR2_EL1_E0POE) : 0);
+ val &= ~(kvm_has_s1pie(vcpu->kvm) ? TCR2_EL1_PIE : 0);
+ return val;
}
return vcpu_read_sys_reg(vcpu, TCR2_EL2);
@@ -1544,7 +1549,7 @@ int __kvm_at_s1e2(struct kvm_vcpu *vcpu, u32 op, u64 vaddr)
* switching context behind everybody's back, disable interrupts...
*/
scoped_guard(write_lock_irqsave, &vcpu->kvm->mmu_lock) {
- u64 val, hcr;
+ u64 val, hcr, tcr2;
bool fail;
val = hcr = read_sysreg(hcr_el2);
@@ -1554,6 +1559,16 @@ int __kvm_at_s1e2(struct kvm_vcpu *vcpu, u32 op, u64 vaddr)
if (!vcpu_el2_e2h_is_set(vcpu))
val |= HCR_EL2_NV | HCR_EL2_NV1;
+ if (cpus_have_final_cap(ARM64_HAS_TCR2)) {
+ tcr2 = read_sysreg_el1(SYS_TCR2);
+ /*
+ * This relies on the reset value of TCR2_EL2
+ * being 0, and not being modified when
+ * FEAT_TCR2 is not exposed to the guest
+ */
+ write_sysreg_el1(vcpu_read_sys_reg(vcpu, TCR2_EL2), SYS_TCR2);
+ }
+
write_sysreg_hcr(val);
isb();
@@ -1577,6 +1592,9 @@ int __kvm_at_s1e2(struct kvm_vcpu *vcpu, u32 op, u64 vaddr)
if (!fail)
par = read_sysreg_par();
+ if (cpus_have_final_cap(ARM64_HAS_TCR2))
+ write_sysreg_el1(tcr2, SYS_TCR2);
+
write_sysreg_hcr(hcr);
isb();
}
--
Without deviation from the norm, progress is not possible.
^ permalink raw reply related [flat|nested] 4+ messages in thread
end of thread, other threads:[~2026-11-16 16:01 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-16 6:00 [PATCH 0/2] KVM: arm64: nv: fix a couple bugs in AT emulation Dan Carpenter
2026-11-16 6:00 ` [PATCH 2/2] KVM: arm64: nv: Update the shadow MMU's VMID for " Dan Carpenter
2026-11-16 6:00 ` [PATCH 1/2] KVM: arm64: nv: Fix AT emulation for TCR2_EL1 Dan Carpenter
2026-11-16 16:01 ` Marc Zyngier
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox