From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1DAE8C433C1 for ; Tue, 30 Mar 2021 17:02:03 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B26A361990 for ; Tue, 30 Mar 2021 17:02:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B26A361990 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Subject:Cc:To: From:Message-ID:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=yhW1bOouZFIGdtdw4zdQZhLnZaQgRFtNe3+ZIdBXtg0=; b=kEO+rcINvTP35FTlvHVz8L+nB we8RKsPshfpNz3CebkOej2yqlHYKscIMzkDb/+f5VVz5ZruWQR/rESNZ3QDtAVi8mki4X0o07USJl el98gWxrWoL1x3IDn04b4pBq2aBK+6CIuoWtgUOe8Rktvlj1eq9re14Kl+mMrjpB/Kh/TLtWS+4PA IKIVXjkmcOPnN7N5cSJAr1LpYb+e3Q2HU4WaiEppLxx5E2d2hHsGVPD5qoC2XX6zqgmsH4bw5KXBg RYHq6ej+29TlBAKjxGPrYIA11By6zhUsL1DZyBvwNz4tznMc1HjfVrdjc4AUuMVcxQgrt/c+lk5HE dyb1ToNBA==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lRHjA-004PId-BI; Tue, 30 Mar 2021 17:00:28 +0000 Received: from mail.kernel.org ([198.145.29.99]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lRHj4-004PI8-5T for linux-arm-kernel@lists.infradead.org; Tue, 30 Mar 2021 17:00:24 +0000 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 98D16619CE; Tue, 30 Mar 2021 17:00:20 +0000 (UTC) Received: from 78.163-31-62.static.virginmediabusiness.co.uk ([62.31.163.78] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94) (envelope-from ) id 1lRHj0-004jUd-KD; Tue, 30 Mar 2021 18:00:18 +0100 Date: Tue, 30 Mar 2021 18:00:18 +0100 Message-ID: <87a6qkr2zh.wl-maz@kernel.org> From: Marc Zyngier To: Will Deacon Cc: linux-arm-kernel@lists.infradead.org, Hector Martin , Arnd Bergmann , Mark Rutland , Catalin Marinas , kernel-team@android.com Subject: Re: [PATCH 2/2] arm64: Cope with CPUs stuck in VHE mode In-Reply-To: <20210329102159.GD3207@willie-the-truck> References: <20210325124721.941182-1-maz@kernel.org> <20210325124721.941182-3-maz@kernel.org> <20210325193318.GD16123@willie-the-truck> <87sg4if9fx.wl-maz@kernel.org> <20210329102159.GD3207@willie-the-truck> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 62.31.163.78 X-SA-Exim-Rcpt-To: will@kernel.org, linux-arm-kernel@lists.infradead.org, marcan@marcan.st, arnd@kernel.org, mark.rutland@arm.com, catalin.marinas@arm.com, kernel-team@android.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210330_180022_740197_C0E9ED2F X-CRM114-Status: GOOD ( 27.65 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Mon, 29 Mar 2021 11:22:00 +0100, Will Deacon wrote: > > On Fri, Mar 26, 2021 at 11:20:18AM +0000, Marc Zyngier wrote: > > I guess I could either make this code conditional on CONFIG_ARM64_VHE > > and let the machine crash early without a word, or have some later > > checks once the machine started booting. In the later case, displaying > > anything useful is going to be a challenge though (the odds of someone > > having a serial console on this box are close to nil). Pick your poison. > > I think the best thing to do would be to fail to initialise KVM if the > kernel is stuck at EL2 but we don't have VHE support compiled in. Is that > do-able? To quote someone, it is "a little ugly on the side". I came up with the following hack. Can't say I'm in love with it, specially the sprinkling of checks in the alternative callbacks, but hey, I can boot the machine without CONFIG_ARM64_VHE, and get the expected splat at boot time: [ 0.033604] ------------[ cut here ]------------ [ 0.033850] CPU: CPUs started in inconsistent modes [ 0.033854] WARNING: CPU: 0 PID: 1 at arch/arm64/kernel/smp.c:434 hyp_mode_check+0x90/0xc4 [ 0.034863] Modules linked in: [ 0.035100] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.12.0-rc3-00103-geee3f110c447-dirty #3231 [ 0.035776] Hardware name: Apple Mac mini (M1, 2020) (DT) [ 0.036192] pstate: 60400009 (nZCv daif +PAN -UAO -TCO BTYPE=--) [ 0.036654] pc : hyp_mode_check+0x90/0xc4 [ 0.036963] lr : hyp_mode_check+0x90/0xc4 [ 0.037271] sp : ffff800010053e30 [ 0.037526] x29: ffff800010053e30 x28: 0000000000000000 [ 0.037935] x27: 0000000000000000 x26: 0000000000000000 [ 0.038344] x25: 0000000000000000 x24: 0000000000000000 [ 0.038754] x23: 0000000000000000 x22: 0000000000000000 [ 0.039163] x21: 0000000000000000 x20: ffffca3b2f53fc04 [ 0.039572] x19: ffffca3b2fac1000 x18: 0000000000000001 [ 0.039981] x17: 00000000cc4379d6 x16: 000000005c7b6156 [ 0.040391] x15: 0000000000000030 x14: ffffffffffffffff [ 0.040800] x13: ffff800090053ab7 x12: ffff800010053ac0 [ 0.041209] x11: 0000000bbe2c6238 x10: ffffca3b2faa0ad8 [ 0.041618] x9 : ffffca3b2e310df0 x8 : fffffffffffe18b8 [ 0.042027] x7 : ffffca3b2fa481d8 x6 : 0000000000002ffd [ 0.042437] x5 : 0000000000000000 x4 : 0000000000000000 [ 0.042846] x3 : 00000000ffffffff x2 : 0000000000000000 [ 0.043255] x1 : 0000000000000000 x0 : ffff4af181631280 [ 0.043665] Call trace: [ 0.043852] hyp_mode_check+0x90/0xc4 [ 0.044134] smp_cpus_done+0x34/0x48 [ 0.044409] smp_init+0x80/0x90 [ 0.044651] kernel_init_freeable+0x108/0x160 [ 0.044986] kernel_init+0x20/0x12c [ 0.045254] ret_from_fork+0x10/0x3c [ 0.045530] ---[ end trace 0736417247c9e9a3 ]--- [...] [ 0.616800] kvm [1]: HYP mode not available I'll wrap that up in a separate patch, and we can then discuss whether we really want it... Thanks, M. diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h index 7379f35ae2c6..69bc4e26aa26 100644 --- a/arch/arm64/include/asm/virt.h +++ b/arch/arm64/include/asm/virt.h @@ -72,6 +72,11 @@ void __hyp_reset_vectors(void); DECLARE_STATIC_KEY_FALSE(kvm_protected_mode_initialized); +static inline bool is_kernel_in_hyp_mode(void) +{ + return read_sysreg(CurrentEL) == CurrentEL_EL2; +} + /* Reports the availability of HYP mode */ static inline bool is_hyp_mode_available(void) { @@ -83,6 +88,10 @@ static inline bool is_hyp_mode_available(void) static_branch_likely(&kvm_protected_mode_initialized)) return true; + /* Catch braindead CPUs */ + if (!IS_ENABLED(CONFIG_ARM64_VHE) && is_kernel_in_hyp_mode()) + return false; + return (__boot_cpu_mode[0] == BOOT_CPU_MODE_EL2 && __boot_cpu_mode[1] == BOOT_CPU_MODE_EL2); } @@ -98,12 +107,11 @@ static inline bool is_hyp_mode_mismatched(void) static_branch_likely(&kvm_protected_mode_initialized)) return false; - return __boot_cpu_mode[0] != __boot_cpu_mode[1]; -} + /* Catch braindead CPUs */ + if (!IS_ENABLED(CONFIG_ARM64_VHE) && is_kernel_in_hyp_mode()) + return true; -static inline bool is_kernel_in_hyp_mode(void) -{ - return read_sysreg(CurrentEL) == CurrentEL_EL2; + return __boot_cpu_mode[0] != __boot_cpu_mode[1]; } static __always_inline bool has_vhe(void) diff --git a/arch/arm64/kvm/va_layout.c b/arch/arm64/kvm/va_layout.c index 978301392d67..edb048654e00 100644 --- a/arch/arm64/kvm/va_layout.c +++ b/arch/arm64/kvm/va_layout.c @@ -156,6 +156,9 @@ void __init kvm_update_va_mask(struct alt_instr *alt, { int i; + if (!is_hyp_mode_available()) + return; + BUG_ON(nr_inst != 5); for (i = 0; i < nr_inst; i++) { @@ -191,6 +194,9 @@ void kvm_patch_vector_branch(struct alt_instr *alt, u64 addr; u32 insn; + if (!is_hyp_mode_available()) + return; + BUG_ON(nr_inst != 4); if (!cpus_have_const_cap(ARM64_SPECTRE_V3A) || WARN_ON_ONCE(has_vhe())) @@ -244,6 +250,9 @@ static void generate_mov_q(u64 val, __le32 *origptr, __le32 *updptr, int nr_inst { u32 insn, oinsn, rd; + if (!is_hyp_mode_available()) + return; + BUG_ON(nr_inst != 4); /* Compute target register */ -- Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel