From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B3D18C77B75 for ; Wed, 17 May 2023 14:13:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Subject:Cc:To:From:Message-ID:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=2gqrFgv7QHhNEIm/vWsygAYOnX/KxJTndg1NLhQkQ00=; b=ZOgGoAogOIqP/d K8WsABshsVgmZMU7U9WOqrK/KXr9jvRPbjiVhvOsZjzIbeoyyJXsckQ+Gn9pwfIAIktKyAHcIR2m+ SPYYXkV7dUr4liKNB5c4JS3PWqCFvgZW6/ROyxFdCti3IzWzRKJZIAgA7JJWOv4KEzo/UXXGrjS3P vLj3Q0DrjkNyMVCe1T/JdumgQwZV16s4ouXMGrZpDZTyApma2TZIL+IYV5dPOeP+C1Jph2pwjgko+ ucqgPdiWcAy+ogRCc9/fgIS5ocVldZ1zctZgUNDXcQC9cKeY2teoqBGDpVrem+427pK8+xX46mSm3 mncZlKtlTEEdV46AD0lw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1pzHtr-00A5ev-1j; Wed, 17 May 2023 14:13:07 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1pzHto-00A5cF-1q for linux-arm-kernel@lists.infradead.org; Wed, 17 May 2023 14:13:06 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 73E0A647B9; Wed, 17 May 2023 14:12:53 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D5BC8C433EF; Wed, 17 May 2023 14:12:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1684332772; bh=w752a/vCH7cdiwc64hg29QXvoNJTSnKFdp+bBQSM6xE=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=XXdc2nRyW1m25T9rU1BsCMR6RDMjAsQ3cJ0vKz3Pxag60PeFJ95zTwVwVgglGz6hc zHmNtHNUHLxRuaGsyoqVDCwx7U5xFmB4CIPIBiHdp9uc+OfnqB7VrvGIG5C4EaRCn+ pFhanLLwvMmX6MYxjoBjm9LeFhTZmRCW+8WN+ymKxuibR+nGXL0bsLaQQJcIzOd4Oj b+tEhZ3UFFEp76f1jiMyYnsFP3eXOAIpac1dsW7iKcOeZBwYbhBD11IHaybofn1MQi qt3mrHFicAYfhZTH2Vlt8xxFUz8j8KL9UVR5BhissB27LEQj3QAO7lxzlH2O4PO09v SNSKV9eI4GYzg== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1pzHta-00FsEi-El; Wed, 17 May 2023 15:12:50 +0100 Date: Wed, 17 May 2023 15:12:50 +0100 Message-ID: <86r0rfkpwd.wl-maz@kernel.org> From: Marc Zyngier To: Eric Auger Cc: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Alexandru Elisei , Andre Przywara , Chase Conklin , Christoffer Dall , Ganapatrao Kulkarni , Darren Hart , Jintack Lim , Russell King , Miguel Luis , James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu Subject: Re: [PATCH v10 00/59] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support In-Reply-To: References: <20230515173103.1017669-1-maz@kernel.org> <16d9fda4-3ead-7d5e-9f54-ef29fbd932ac@redhat.com> <87zg64nhqh.wl-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/28.2 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: eauger@redhat.com, kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, alexandru.elisei@arm.com, andre.przywara@arm.com, chase.conklin@arm.com, christoffer.dall@arm.com, gankulkarni@os.amperecomputing.com, darren@os.amperecomputing.com, jintack@cs.columbia.edu, rmk+kernel@armlinux.org.uk, miguel.luis@oracle.com, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230517_071304_715451_C31C5576 X-CRM114-Status: GOOD ( 41.67 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Wed, 17 May 2023 09:59:45 +0100, Eric Auger wrote: > > Hi Marc, > > On 5/16/23 22:28, Marc Zyngier wrote: > > On Tue, 16 May 2023 17:53:14 +0100, > > Eric Auger wrote: > >> > >> Hi Marc, > >> > >> On 5/15/23 19:30, Marc Zyngier wrote: > >>> This is the 4th drop of NV support on arm64 for this year. > >>> > >>> For the previous episodes, see [1]. > >>> > >>> What's changed: > >>> > >>> - New framework to track system register traps that are reinjected in > >>> guest EL2. It is expected to replace the discrete handling we have > >>> enjoyed so far, which didn't scale at all. This has already fixed a > >>> number of bugs that were hidden (a bunch of traps were never > >>> forwarded...). Still a work in progress, but this is going in the > >>> right direction. > >>> > >>> - Allow the L1 hypervisor to have a S2 that has an input larger than > >>> the L0 IPA space. This fixes a number of subtle issues, depending on > >>> how the initial guest was created. > >>> > >>> - Consequently, the patch series has gone longer again. Boo. But > >>> hopefully some of it is easier to review... > >>> > >>> [1] https://lore.kernel.org/r/20230405154008.3552854-1-maz@kernel.org > >> > >> I have started testing this and when booting my fedora guest I get > >> > >> [ 151.796544] kvm [7617]: Unsupported guest sys_reg access at: > >> 23f425fd0 [80000209] > >> [ 151.796544] { Op0( 3), Op1( 3), CRn(14), CRm( 3), Op2( 1), func_write }, > >> > >> as soon as the host has kvm-arm.mode=nested > >> > >> This seems to be triggered very early by EDK2 > >> (ArmPkg/Drivers/TimerDxe/TimerDxe.c). > >> > >> If I am not wrong this CNTV_CTL_EL0. Do you have any idea? > > > > So here's my current analysis: > > > > I assume you are running EDK2 as the L1 guest in a nested > > configuration. I also assume that you are not running on an Apple > > CPU. If these assumptions are correct, then EDK2 runs at vEL2, and is > > in nVHE mode. > > > > Finally, I'm going to assume that your implementation has FEAT_ECV and > > FEAT_NV2, because I can't see how it could fail otherwise. > all the above is correct. > > > > In these precise conditions, KVM sets the CNTHCTL_EL2.EL1TVT bit so > > that we can trap the EL0 virtual timer and faithfully emulate it (it > > is otherwise written to memory, which isn't very helpful). > > indeed > > > > As it turns out, we don't handle these traps. I didn't spot it because > > my test machines are all Apple boxes that don't have a nVHE mode, so > > nothing on the nVHE path is getting *ANY* coverage. Hint: having > > access to such a machine would help (shipping address on request!). > > Otherwise, I'll eventually kill the nVHE support altogether. > > > > I have written the following patch, which compiles, but that I cannot > > test with my current setup. Could you please give it a go? > > with the patch below, my guest boots nicely. You did it great on the 1st > shot!!! So this fixes my issue. I will continue testing the v10. Thanks a lot for reporting the issue and testing my hacks. I'll eventually fold it into the rest of the series. By the way, what are you using as your VMM? I'd really like to reproduce your setup. Cheers, M. -- Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel