Re: [PATCH 0/3] KVM: arm64: Load the stage-2 MMU from vcpu_load() for VHE

[Marc Zyngier <maz@kernel.org>]

On Fri, 06 Oct 2023 12:13:00 +0100,
Marc Zyngier <maz@kernel.org> wrote:
> 
> On Fri, 06 Oct 2023 10:35:57 +0100,
> Oliver Upton <oliver.upton@linux.dev> wrote:
> > 
> > Unlike nVHE, there is no need to switch the stage-2 MMU around on guest
> > entry/exit in VHE mode as the host is running at EL2. Despite this KVM
> > reloads the stage-2 on every guest entry, which is needless.
> > 
> > This series moves the setup of the stage-2 MMU context to vcpu_load()
> > when running in VHE mode. This is likely to be a win across the board,
> > but also allows us to remove an ISB on the guest entry path for systems
> > with one of the speculative AT errata.
> > 
> > None of my machines affected by the AT errata are VHE-capable, so it'd
> > be appreciated if someone could give this series a go and make sure I
> > haven't wrecked anything.
> 
> It totally breaks on my A55 board. Running a single guest seems OK,
> but running a number of the concurrently makes them explode early on
> (faults in EFI...)
> 
> I guess we end-up running with the wrong VTTBR at times, which would
> be interesting...

Fun fact:

diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index b0cafd7c5f8f..40c84db5884a 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -195,6 +195,11 @@ static int __kvm_vcpu_run_vhe(struct kvm_vcpu *vcpu)
 	sysreg_restore_guest_state_vhe(guest_ctxt);
 	__debug_switch_to_guest(vcpu);
 
+	WARN_ONCE(kvm_get_vttbr(vcpu->arch.hw_mmu) != read_sysreg(vttbr_el2),
+		  "Oh crap %llx vs %llx\n",
+		  kvm_get_vttbr(vcpu->arch.hw_mmu),
+		  read_sysreg(vttbr_el2));
+
 	if (is_hyp_ctxt(vcpu))
 		vcpu_set_flag(vcpu, VCPU_HYP_CONTEXT);
 	else

[   36.190355] Oh crap 10000057a6001 vs 57a6001

My bet is that the VMID isn't allocated on first load, and everything
goes downhill from there.

	M.

-- 
Without deviation from the norm, progress is not possible.