Re: [PATCH v11 00/43] KVM: arm64: Nested Virtualization support (FEAT_NV2 only)

From: Marc Zyngier <maz@kernel.org>
To: Ganapatrao Kulkarni <gankulkarni@os.amperecomputing.com>
Cc: Miguel Luis <miguel.luis@oracle.com>,
	"kvmarm@lists.linux.dev" <kvmarm@lists.linux.dev>,
	"kvm@vger.kernel.org" <kvm@vger.kernel.org>,
	"linux-arm-kernel@lists.infradead.org"
	<linux-arm-kernel@lists.infradead.org>,
	Alexandru Elisei <alexandru.elisei@arm.com>,
	Andre Przywara <andre.przywara@arm.com>,
	Chase Conklin <chase.conklin@arm.com>,
	Christoffer Dall <christoffer.dall@arm.com>,
	Darren Hart <darren@os.amperecomputing.com>,
	Jintack Lim <jintack@cs.columbia.edu>,
	Russell King <rmk+kernel@armlinux.org.uk>,
	James Morse <james.morse@arm.com>,
	Suzuki K Poulose <suzuki.poulose@arm.com>,
	Oliver Upton <oliver.upton@linux.dev>,
	Zenghui Yu <yuzenghui@huawei.com>
Subject: Re: [PATCH v11 00/43] KVM: arm64: Nested Virtualization support (FEAT_NV2 only)
Date: Mon, 27 Nov 2023 09:22:00 +0000	[thread overview]
Message-ID: <86jzq3d007.wl-maz@kernel.org> (raw)
In-Reply-To: <9f8656c7-8036-4bd6-a0f5-4fa531f95b2f@os.amperecomputing.com>

On Mon, 27 Nov 2023 07:26:58 +0000,
Ganapatrao Kulkarni <gankulkarni@os.amperecomputing.com> wrote:
> 
> 
> 
> On 24-11-2023 08:02 pm, Marc Zyngier wrote:
> > On Fri, 24 Nov 2023 13:22:22 +0000,
> > Ganapatrao Kulkarni <gankulkarni@os.amperecomputing.com> wrote:
> >> 
> >>> How is this value possible if the write to HCR_EL2 has taken place?
> >>> When do you sample this?
> >> 
> >> I am not sure how and where it got set. I think, whatever it is set,
> >> it is due to false return of vcpu_el2_e2h_is_set(). Need to
> >> understand/debug.
> >> The vhcr_el2 value I have shared is traced along with hcr in function
> >> __activate_traps/__compute_hcr.
> > 
> > Here's my hunch:
> > 
> > The guest boots with E2H=0, because we don't advertise anything else
> > on your HW. So we run with NV1=1 until we try to *upgrade* to VHE. NV2
> > means that HCR_EL2 is writable (to memory) without a trap. But we're
> > still running with NV1=1.
> > 
> > Subsequently, we access a sysreg that should never trap for a VHE
> > guest, but we're with the wrong config. Bad things happen.
> > 
> > Unfortunately, NV2 is pretty much incompatible with E2H being updated,
> > because it cannot perform the changes that this would result into at
> > the point where they should happen. We can try and do a best effort
> > handling, but you can always trick it.
> > 
> > Anyway, can you see if the hack below helps? I'm not keen on it at
> > all, but this would be a good data point.
> 
> Thanks Marc, this diff fixes the issue.
> Just wondering what is changed w.r.t to L1 handling from V10 to V11
> that it requires this trick?

Not completely sure. Before v11, anything that would trap would be
silently handled by the FEAT_NV code. Now, a trap for something that
is supposed to be redirected to VNCR results in an UNDEF exception.

I suspect that the exception is handled again as a call to
__finalise_el2(), probably because the write to VBAR_EL1 didn't do
what it was supposed to do.

> Also why this was not seen on your platform, is it E2H0 enabled?

It doesn't have FEAT_E2H0, and that's the whole point. No E2H0, no
problems, as the guest cannot trick the host into losing track of the
state (which I'm pretty sure can happen even with this ugly hack).

I will probably completely disable NV1 support in the next drop, and
make NV support only VHE guests. Which is the only mode that makes any
sense anyway.

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel