From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BCCFEC77B7F for ; Tue, 16 May 2023 20:29:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Subject:Cc:To:From:Message-ID:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ZoV0+Xuj8KNBofPKugRNRI3UfNnfLfvcg2ZSZ9DDGnM=; b=K2J8tqfIcTGP6r ZY3zYFuMmhavCk6CWJ9/cy5TkqwqATAIPjBKch2zTKmmt10t0XtcrbdVX74rAa8ug3aM7zrDxjwja t4WvKzaq8C602MwHi5x7QOUg+G4XS9Zhu092mhS0JsAQ7K/9k2oFZuUrY8zXHC1sSBX8VDZ+hTfli XjqQNECmcMy1SeDn9P0qjMyS8jH9ggkZR9GSLNxwHpiQZHe7pjnWjkDjAjagq3siUFD2mW4XgBKrn Db4bverCMr4AR2epTA4Zkazs2F8+NkHnZR30dI6tQLVDxithtzYrsjCKpb43/lR7uCafU2mpq5YE+ PMOGpy8N0Uo/f2swxuWQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1pz1Hr-0073X4-2x; Tue, 16 May 2023 20:28:47 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1pz1Ho-0073Wd-1s for linux-arm-kernel@lists.infradead.org; Tue, 16 May 2023 20:28:46 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 130446381E; Tue, 16 May 2023 20:28:44 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 766F7C433D2; Tue, 16 May 2023 20:28:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1684268923; bh=DW61ZP82H+VuUq1yDKIyiiIILGwHFpx8XnUhpoPXlo0=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=MZrqtQmawO8EHlXUlLZjKDgVjH2GnVnBRWadDqRQR/XQ4Qkxn/sGH9gnbijtf92eb mkZKNEX5HpqXeEW2gfAUcUcGjB42ZNeSF+p5slfbMl9H0y2n8OfogIV902gTRJ7dus r8k0nxRq97EbEte5W5QnSqYkyJMlRGczh05PoE8N2L0ZlaT37RR5+hCTYZjWxP7PWm 7dbJ0SLaewRgwaeWexA2XQQzE9lmHvlZN69gjp1NmQd1ARv8meST/vetdJI+hc8kDI w+xXPO9oH4roVxQRUjElEyyn216ndLkm4g6AtCd+HhiuJvaCL5E0KKM+OoUOLqAdKy 5xkkPvdqXk6ZA== Received: from 82-132-215-232.dab.02.net ([82.132.215.232] helo=wait-a-minute.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1pz1Hk-00FfLo-OV; Tue, 16 May 2023 21:28:41 +0100 Date: Tue, 16 May 2023 21:28:38 +0100 Message-ID: <87zg64nhqh.wl-maz@kernel.org> From: Marc Zyngier To: Eric Auger Cc: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Alexandru Elisei , Andre Przywara , Chase Conklin , Christoffer Dall , Ganapatrao Kulkarni , Darren Hart , Jintack Lim , Russell King , Miguel Luis , James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu Subject: Re: [PATCH v10 00/59] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support In-Reply-To: <16d9fda4-3ead-7d5e-9f54-ef29fbd932ac@redhat.com> References: <20230515173103.1017669-1-maz@kernel.org> <16d9fda4-3ead-7d5e-9f54-ef29fbd932ac@redhat.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/28.2 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 82.132.215.232 X-SA-Exim-Rcpt-To: eauger@redhat.com, kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, alexandru.elisei@arm.com, andre.przywara@arm.com, chase.conklin@arm.com, christoffer.dall@arm.com, gankulkarni@os.amperecomputing.com, darren@os.amperecomputing.com, jintack@cs.columbia.edu, rmk+kernel@armlinux.org.uk, miguel.luis@oracle.com, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230516_132844_703660_F3837F61 X-CRM114-Status: GOOD ( 37.86 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Tue, 16 May 2023 17:53:14 +0100, Eric Auger wrote: > > Hi Marc, > > On 5/15/23 19:30, Marc Zyngier wrote: > > This is the 4th drop of NV support on arm64 for this year. > > > > For the previous episodes, see [1]. > > > > What's changed: > > > > - New framework to track system register traps that are reinjected in > > guest EL2. It is expected to replace the discrete handling we have > > enjoyed so far, which didn't scale at all. This has already fixed a > > number of bugs that were hidden (a bunch of traps were never > > forwarded...). Still a work in progress, but this is going in the > > right direction. > > > > - Allow the L1 hypervisor to have a S2 that has an input larger than > > the L0 IPA space. This fixes a number of subtle issues, depending on > > how the initial guest was created. > > > > - Consequently, the patch series has gone longer again. Boo. But > > hopefully some of it is easier to review... > > > > [1] https://lore.kernel.org/r/20230405154008.3552854-1-maz@kernel.org > > I have started testing this and when booting my fedora guest I get > > [ 151.796544] kvm [7617]: Unsupported guest sys_reg access at: > 23f425fd0 [80000209] > [ 151.796544] { Op0( 3), Op1( 3), CRn(14), CRm( 3), Op2( 1), func_write }, > > as soon as the host has kvm-arm.mode=nested > > This seems to be triggered very early by EDK2 > (ArmPkg/Drivers/TimerDxe/TimerDxe.c). > > If I am not wrong this CNTV_CTL_EL0. Do you have any idea? So here's my current analysis: I assume you are running EDK2 as the L1 guest in a nested configuration. I also assume that you are not running on an Apple CPU. If these assumptions are correct, then EDK2 runs at vEL2, and is in nVHE mode. Finally, I'm going to assume that your implementation has FEAT_ECV and FEAT_NV2, because I can't see how it could fail otherwise. In these precise conditions, KVM sets the CNTHCTL_EL2.EL1TVT bit so that we can trap the EL0 virtual timer and faithfully emulate it (it is otherwise written to memory, which isn't very helpful). As it turns out, we don't handle these traps. I didn't spot it because my test machines are all Apple boxes that don't have a nVHE mode, so nothing on the nVHE path is getting *ANY* coverage. Hint: having access to such a machine would help (shipping address on request!). Otherwise, I'll eventually kill the nVHE support altogether. I have written the following patch, which compiles, but that I cannot test with my current setup. Could you please give it a go? Thanks again, M. >From feb03b57de0bcb83254a2d6a3ce320f5e39434b6 Mon Sep 17 00:00:00 2001 From: Marc Zyngier Date: Tue, 16 May 2023 21:06:20 +0100 Subject: [PATCH] KVM: arm64: Handle virtual timer traps when CNTHCTL_EL2.EL1TVT is set Signed-off-by: Marc Zyngier --- arch/arm64/include/asm/sysreg.h | 1 + arch/arm64/kvm/sys_regs.c | 28 ++++++++++++++++++++++++++++ 2 files changed, 29 insertions(+) diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h index 72ff6df5d75b..77a61179ea37 100644 --- a/arch/arm64/include/asm/sysreg.h +++ b/arch/arm64/include/asm/sysreg.h @@ -436,6 +436,7 @@ #define SYS_CNTP_CTL_EL0 sys_reg(3, 3, 14, 2, 1) #define SYS_CNTP_CVAL_EL0 sys_reg(3, 3, 14, 2, 2) +#define SYS_CNTV_TVAL_EL0 sys_reg(3, 3, 14, 3, 0) #define SYS_CNTV_CTL_EL0 sys_reg(3, 3, 14, 3, 1) #define SYS_CNTV_CVAL_EL0 sys_reg(3, 3, 14, 3, 2) diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c index 27a29dcbfcd2..9aa9c4e4b4d6 100644 --- a/arch/arm64/kvm/sys_regs.c +++ b/arch/arm64/kvm/sys_regs.c @@ -1328,6 +1328,14 @@ static bool access_arch_timer(struct kvm_vcpu *vcpu, treg = TIMER_REG_TVAL; break; + case SYS_CNTV_TVAL_EL0: + if (is_hyp_ctxt(vcpu) && vcpu_el2_e2h_is_set(vcpu)) + tmr = TIMER_HVTIMER; + else + tmr = TIMER_VTIMER; + treg = TIMER_REG_TVAL; + break; + case SYS_AARCH32_CNTP_TVAL: case SYS_CNTP_TVAL_EL02: tmr = TIMER_PTIMER; @@ -1357,6 +1365,14 @@ static bool access_arch_timer(struct kvm_vcpu *vcpu, treg = TIMER_REG_CTL; break; + case SYS_CNTV_CTL_EL0: + if (is_hyp_ctxt(vcpu) && vcpu_el2_e2h_is_set(vcpu)) + tmr = TIMER_HVTIMER; + else + tmr = TIMER_VTIMER; + treg = TIMER_REG_CTL; + break; + case SYS_AARCH32_CNTP_CTL: case SYS_CNTP_CTL_EL02: tmr = TIMER_PTIMER; @@ -1386,6 +1402,14 @@ static bool access_arch_timer(struct kvm_vcpu *vcpu, treg = TIMER_REG_CVAL; break; + case SYS_CNTV_CVAL_EL0: + if (is_hyp_ctxt(vcpu) && vcpu_el2_e2h_is_set(vcpu)) + tmr = TIMER_HVTIMER; + else + tmr = TIMER_VTIMER; + treg = TIMER_REG_CVAL; + break; + case SYS_AARCH32_CNTP_CVAL: case SYS_CNTP_CVAL_EL02: tmr = TIMER_PTIMER; @@ -2510,6 +2534,10 @@ static const struct sys_reg_desc sys_reg_descs[] = { { SYS_DESC(SYS_CNTP_CTL_EL0), access_arch_timer }, { SYS_DESC(SYS_CNTP_CVAL_EL0), access_arch_timer }, + { SYS_DESC(SYS_CNTV_TVAL_EL0), access_arch_timer }, + { SYS_DESC(SYS_CNTV_CTL_EL0), access_arch_timer }, + { SYS_DESC(SYS_CNTV_CVAL_EL0), access_arch_timer }, + /* PMEVCNTRn_EL0 */ PMU_PMEVCNTR_EL0(0), PMU_PMEVCNTR_EL0(1), -- 2.39.2 -- Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel