From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A7479C433FE for ; Tue, 11 Oct 2022 18:49:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=5clyebg6yW1qUspbAdJ4g1jjF9BCxvahbyweocWjqXQ=; b=rHkhetiXOX3jfu coGivrZy08zUGn1ZvVe3mY+Unont0xOyBTKuUIhvd3PkBoDdUwMO1+Q0ekh8zN9yamEjtdJaB4F4l azsR0Lr6aDTUC0p7biljlLXpszMA/t83cE+H0FQ6k/dVYv/Le4tHAlwaxYyO9otPzDM8IFxABjrnt VPqr8uhnnoy4OWs8nlkAnn2GuF1ejGFLq8090Wx5CqrvAZG069rebTVyn4yL0z2IhB7dfhJ2SZ0eo mf1FOP82gTm1HSum7ADrZ0gz4+XKnirjDxllxdTElQNp73FscPoGd++P5CVw9L5mf8q+LraPaA2wm P17OzaTgNdpbD0lpVu2Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oiKJD-005VZh-Kt; Tue, 11 Oct 2022 18:48:55 +0000 Received: from out2.migadu.com ([2001:41d0:2:aacc::]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oiKJ9-005VVj-Av for linux-arm-kernel@lists.infradead.org; Tue, 11 Oct 2022 18:48:53 +0000 Date: Tue, 11 Oct 2022 11:48:39 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1665514124; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=jwUuFfKVT/IBhZS1xJd712tVZ2U42EWvKxBAY1OBy2U=; b=WoLRw3kDbRW+fd1QBnPOGS0PV3qDYDtz+HNnnM49JtYpz9z0NauyT7w8dI/u8mAN6dibh2 9YWmQPSmt1jq2O+ny0GVym+UvNC2WvHXII7qkKl7YRVraFvjCqILW4NFYAfgIq3Y0v8Co5 mzRtohfWXsNWlfosYCl1vGXLpTRuiys= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Oliver Upton To: Marc Zyngier Cc: kvmarm@lists.cs.columbia.edu, kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, James Morse , Suzuki K Poulose , Alexandru Elisei , Quentin Perret , Will Deacon , Vincent Donnefort Subject: Re: [PATCH] KVM: arm64: pkvm: Fixup boot mode to reflect that the kernel resumes from EL1 Message-ID: References: <20221011165400.1241729-1-maz@kernel.org> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20221011165400.1241729-1-maz@kernel.org> X-Migadu-Flow: FLOW_OUT X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221011_114851_893043_2EDE4B2D X-CRM114-Status: GOOD ( 27.66 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Tue, Oct 11, 2022 at 05:54:00PM +0100, Marc Zyngier wrote: > The kernel has an awfully complicated boot sequence in order to cope > with the various EL2 configurations, including those that "enhanced" > the architecture. We go from EL2 to EL1, then back to EL2, staying > at EL2 if VHE capable and otherwise go back to EL1. > > Here's a paracetamol tablet for you. Heh, still have a bit of a headache from this :) I'm having a hard time following where we skip the EL2 promotion based on __boot_cpu_mode. On the cpu_resume() path it looks like we take the return of init_kernel_el() and pass that along to finalise_el2(). As we are in EL1 at this point, it seems like we'd go init_kernel_el() -> init_el1(). What am I missing? -- Thanks, Oliver > The cpu_resume path follows the same logic, because coming up with > two versions of a square wheel is hard. > > However, things aren't this straightforward with pKVM, as the host > resume path is always proxied by the hypervisor, which means that > the kernel is always entered at EL1. Which contradicts what the > __boot_cpu_mode[] array contains (it obviously says EL2). > > This thus triggers a HVC call from EL1 to EL2 in a vain attempt > to upgrade from EL1 to EL2 VHE, which we are, funnily enough, > reluctant to grant to the host kernel. This is also completely > unexpected, and puzzles your average EL2 hacker. > > Address it by fixing up the boot mode at the point the host gets > deprivileged. is_hyp_mode_available() and co already have a static > branch to deal with this, making it pretty safe. > > Reported-by: Vincent Donnefort > Signed-off-by: Marc Zyngier > --- > arch/arm64/kvm/arm.c | 11 +++++++++++ > 1 file changed, 11 insertions(+) > > diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c > index b6c9bfa8492f..cf075c9b9ab1 100644 > --- a/arch/arm64/kvm/arm.c > +++ b/arch/arm64/kvm/arm.c > @@ -2107,6 +2107,17 @@ static int pkvm_drop_host_privileges(void) > * once the host stage 2 is installed. > */ > static_branch_enable(&kvm_protected_mode_initialized); > + > + /* > + * Fixup the boot mode so that we don't take spurious round > + * trips via EL2 on cpu_resume. Flush to the PoC for a good > + * measure, so that it can be observed by a CPU coming out of > + * suspend with the MMU off. > + */ > + __boot_cpu_mode[0] = __boot_cpu_mode[1] = BOOT_CPU_MODE_EL1; > + dcache_clean_poc((unsigned long)__boot_cpu_mode, > + (unsigned long)(__boot_cpu_mode + 2)); > + > on_each_cpu(_kvm_host_prot_finalize, &ret, 1); > return ret; > } > -- > 2.34.1 > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel