From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A7305C0755A for ; Mon, 27 Nov 2023 11:46:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Subject:Cc:To:From:Message-ID:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=3GZKDl3p3/9Au0i9PY4N52MOWVTcORJXO+COn8/Btsw=; b=ouA57L1fDwnNAU Eb3U07EESaAhj44TVqUpPckIDqTAcBzhqJTj9+zISHlLazwHuEEdgX1oPW/uSQoPKz94mk9P6SJ2l bxRFS0IwLgDKXEr2NsDNzwgobJ7LfL6s5yHALGNNGR4KP5gOvUIqG96VZ2WPETuNp7RBT7krzOMCn /ZbiXCGIgdh8YMkUWIeOB8FdQk4EuH/wHLBzmLqDVd9aVN/1PearQ+uM3AS6zIBRmheAZtggo18uJ mnhnu8AIWtH0y0CzWNEf8/7pa+gdfmhJPaN9jK7JWaFNSChm2FMxomSPrlW6tDqg7NI1koZ29xooh o6rjMpWvxymMYJIrstlw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1r7a3h-002Hq1-0e; Mon, 27 Nov 2023 11:45:49 +0000 Received: from ams.source.kernel.org ([2604:1380:4601:e00::1]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1r7a3e-002Hna-0k for linux-arm-kernel@lists.infradead.org; Mon, 27 Nov 2023 11:45:47 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by ams.source.kernel.org (Postfix) with ESMTP id BDCEFB82CA9; Mon, 27 Nov 2023 11:45:44 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 20791C433C7; Mon, 27 Nov 2023 11:45:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1701085544; bh=fR7axwwaqdOnZ7+hJpsO31cSOdlPA/4ws0EQLkXICwM=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=TBT467QjtZ/emVrJAaWgmbMkUb/5qwCubJokavxLNgqIOwvX6HCGspUnvcE+UYSi5 Jj9fQ/Z4YVPrLjA52L3vtlR5/dh48Zv+ULQhcVKM3mI8VSqzsYTYbqwUrIDq6z7Fsl 1XSj1FG5pUbKIWOAXQowWwSxL1mrF2QbSKrs93G2W6Fn9Khpf9VV+59tWMUROL3ZJ/ LHzkhv+e+7126p+R5R+Vpr8QHz5TuyzyjJZIFi/6kne2+84vqWboQWHtFAbpNFGNkT hB8o1kmA9ILWUTE+DFBD4jd22+Sd4SU4593if0YuuIg05yJR3xzlIYXQ+FqHnDzlwp f34yubxBLkg+Q== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1r7a3Z-00Gldl-Ck; Mon, 27 Nov 2023 11:45:41 +0000 Date: Mon, 27 Nov 2023 11:45:40 +0000 Message-ID: <86fs0rctcr.wl-maz@kernel.org> From: Marc Zyngier To: Ganapatrao Kulkarni Cc: Miguel Luis , "kvmarm@lists.linux.dev" , "kvm@vger.kernel.org" , "linux-arm-kernel@lists.infradead.org" , Alexandru Elisei , Andre Przywara , Chase Conklin , Christoffer Dall , Darren Hart , Jintack Lim , Russell King , James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu Subject: Re: [PATCH v11 00/43] KVM: arm64: Nested Virtualization support (FEAT_NV2 only) In-Reply-To: <1fe79744-f8a4-466f-8f1a-32e6fab78be9@os.amperecomputing.com> References: <20231120131027.854038-1-maz@kernel.org> <86msv7ylnu.wl-maz@kernel.org> <05733774-4210-4097-9912-fb3aa8542fdd@oracle.com> <86a5r4zafh.wl-maz@kernel.org> <134912e4-beed-4ab6-8ce1-33e69ec382b3@os.amperecomputing.com> <868r6nzc5y.wl-maz@kernel.org> <65dc2a93-0a17-4433-b3a5-430bf516ffe9@os.amperecomputing.com> <86o7fjco13.wl-maz@kernel.org> <86leancjcr.wl-maz@kernel.org> <9f8656c7-8036-4bd6-a0f5-4fa531f95b2f@os.amperecomputing.com> <86jzq3d007.wl-maz@kernel.org> <1fe79744-f8a4-466f-8f1a-32e6fab78be9@os.amperecomputing.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/29.1 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: gankulkarni@os.amperecomputing.com, miguel.luis@oracle.com, kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, alexandru.elisei@arm.com, andre.przywara@arm.com, chase.conklin@arm.com, christoffer.dall@arm.com, darren@os.amperecomputing.com, jintack@cs.columbia.edu, rmk+kernel@armlinux.org.uk, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231127_034546_564420_09B2BC0F X-CRM114-Status: GOOD ( 39.44 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Mon, 27 Nov 2023 10:59:36 +0000, Ganapatrao Kulkarni wrote: > > > > On 27-11-2023 02:52 pm, Marc Zyngier wrote: > > On Mon, 27 Nov 2023 07:26:58 +0000, > > Ganapatrao Kulkarni wrote: > >> > >> > >> > >> On 24-11-2023 08:02 pm, Marc Zyngier wrote: > >>> On Fri, 24 Nov 2023 13:22:22 +0000, > >>> Ganapatrao Kulkarni wrote: > >>>> > >>>>> How is this value possible if the write to HCR_EL2 has taken place? > >>>>> When do you sample this? > >>>> > >>>> I am not sure how and where it got set. I think, whatever it is set, > >>>> it is due to false return of vcpu_el2_e2h_is_set(). Need to > >>>> understand/debug. > >>>> The vhcr_el2 value I have shared is traced along with hcr in function > >>>> __activate_traps/__compute_hcr. > >>> > >>> Here's my hunch: > >>> > >>> The guest boots with E2H=0, because we don't advertise anything else > >>> on your HW. So we run with NV1=1 until we try to *upgrade* to VHE. NV2 > >>> means that HCR_EL2 is writable (to memory) without a trap. But we're > >>> still running with NV1=1. > >>> > >>> Subsequently, we access a sysreg that should never trap for a VHE > >>> guest, but we're with the wrong config. Bad things happen. > >>> > >>> Unfortunately, NV2 is pretty much incompatible with E2H being updated, > >>> because it cannot perform the changes that this would result into at > >>> the point where they should happen. We can try and do a best effort > >>> handling, but you can always trick it. > >>> > >>> Anyway, can you see if the hack below helps? I'm not keen on it at > >>> all, but this would be a good data point. > >> > >> Thanks Marc, this diff fixes the issue. > >> Just wondering what is changed w.r.t to L1 handling from V10 to V11 > >> that it requires this trick? > > > > Not completely sure. Before v11, anything that would trap would be > > silently handled by the FEAT_NV code. Now, a trap for something that > > is supposed to be redirected to VNCR results in an UNDEF exception. > > > > I suspect that the exception is handled again as a call to > > __finalise_el2(), probably because the write to VBAR_EL1 didn't do > > what it was supposed to do. > > > >> Also why this was not seen on your platform, is it E2H0 enabled? > > > > It doesn't have FEAT_E2H0, and that's the whole point. No E2H0, no > > problems, as the guest cannot trick the host into losing track of the > > state (which I'm pretty sure can happen even with this ugly hack). > > > > I will probably completely disable NV1 support in the next drop, and > > make NV support only VHE guests. Which is the only mode that makes any > > sense anyway. > > > > Thanks, absolutely makes sense to have *VHE-only* L1, looking forward > to a next drop. Note that this won't be restricted to L1, but will affect *everything. No non-VHE guest will be supported at any level whatsoever, and NV will always expose ID_AA64MMFR4_EL1.E2H0=0b1110, indicating that HCR_EL2.NV1 is RES0, on top of ID_AA64MMFR4_EL1.NV_frac=1 (NV2 only). M. -- Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel