From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A6B62199B3; Fri, 24 Nov 2023 10:19:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="szTiw9F2" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1B394C433C8; Fri, 24 Nov 2023 10:19:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1700821164; bh=ezJrSp23KXb/hO6ks68IIX+fbvqWMNg+9nVoAxoKW8Q=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=szTiw9F2cGc2s71ZDdtpxoFF0EjWH2OY7N5AfnoWEE8zubiqo3Zn+JxLg7za7mO0p t1HV088SZOdi7zVKtTSwy2B/WJtDOXUyHQ4gLpzWm+9gi5B9Ih1rTTy1D0SdPOpwtT y0vjOpAuo08SS4pEyN2RL8Neyoj53swr+5je3F6etgitdJmzihA6SJntl/04QpCTQk RlzJ5ewHqXd/cpcRESrgv8emSxVUSEG34zj96iIOgcjK0L9G/lA7pBibiobDA1uryw pHTGB3TDAkEbxK+ulSe/wZFM5BC9vOZtQfYrdZYegvXd3tjM2KlCxsbXtwtU899BXC BHcf7Bh+ddkwg== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1r6THN-00G3Ie-Jc; Fri, 24 Nov 2023 10:19:21 +0000 Date: Fri, 24 Nov 2023 10:19:21 +0000 Message-ID: <868r6nzc5y.wl-maz@kernel.org> From: Marc Zyngier To: Ganapatrao Kulkarni Cc: Miguel Luis , "kvmarm@lists.linux.dev" , "kvm@vger.kernel.org" , "linux-arm-kernel@lists.infradead.org" , Alexandru Elisei , Andre Przywara , Chase Conklin , Christoffer Dall , Darren Hart , Jintack Lim , Russell King , James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu Subject: Re: [PATCH v11 00/43] KVM: arm64: Nested Virtualization support (FEAT_NV2 only) In-Reply-To: <134912e4-beed-4ab6-8ce1-33e69ec382b3@os.amperecomputing.com> References: <20231120131027.854038-1-maz@kernel.org> <86msv7ylnu.wl-maz@kernel.org> <05733774-4210-4097-9912-fb3aa8542fdd@oracle.com> <86a5r4zafh.wl-maz@kernel.org> <134912e4-beed-4ab6-8ce1-33e69ec382b3@os.amperecomputing.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/29.1 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) Precedence: bulk X-Mailing-List: kvmarm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: gankulkarni@os.amperecomputing.com, miguel.luis@oracle.com, kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, alexandru.elisei@arm.com, andre.przywara@arm.com, chase.conklin@arm.com, christoffer.dall@arm.com, darren@os.amperecomputing.com, jintack@cs.columbia.edu, rmk+kernel@armlinux.org.uk, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false On Fri, 24 Nov 2023 09:50:33 +0000, Ganapatrao Kulkarni wrote: >=20 >=20 >=20 > On 23-11-2023 10:14 pm, Marc Zyngier wrote: > > On Thu, 23 Nov 2023 16:21:48 +0000, > > Miguel Luis wrote: > >>=20 > >> Hi Marc, > >>=20 > >> On 21/11/2023 18:02, Marc Zyngier wrote: > >>> On Tue, 21 Nov 2023 16:49:52 +0000, > >>> Miguel Luis wrote: > >>>> Hi Marc, > >>>>=20 > >>>>> On 20 Nov 2023, at 12:09, Marc Zyngier wrote: > >>>>>=20 > >>>>> This is the 5th drop of NV support on arm64 for this year, and most > >>>>> probably the last one for this side of Christmas. > >>>>>=20 > >>>>> For the previous episodes, see [1]. > >>>>>=20 > >>>>> What's changed: > >>>>>=20 > >>>>> - Drop support for the original FEAT_NV. No existing hardware suppo= rts > >>>>> it without FEAT_NV2, and the architecture is deprecating the form= er > >>>>> entirely. This results in fewer patches, and a slightly simpler > >>>>> model overall. > >>>>>=20 > >>>>> - Reorganise the series to make it a bit more logical now that FEAT= _NV > >>>>> is gone. > >>>>>=20 > >>>>> - Apply the NV idreg restrictions on VM first run rather than on ea= ch > >>>>> access. > >>>>>=20 > >>>>> - Make the nested vgic shadow CPU interface a per-CPU structure rat= her > >>>>> than per-vcpu. > >>>>>=20 > >>>>> - Fix the EL0 timer fastpath > >>>>>=20 > >>>>> - Work around the architecture deficiencies when trapping WFI from a > >>>>> L2 guest. > >>>>>=20 > >>>>> - Fix sampling of nested vgic state (MISR, ELRSR, EISR) > >>>>>=20 > >>>>> - Drop the patches that have already been merged (NV trap forwardin= g, > >>>>> per-MMU VTCR) > >>>>>=20 > >>>>> - Rebased on top of 6.7-rc2 + the FEAT_E2H0 support [2]. > >>>>>=20 > >>>>> The branch containing these patches (and more) is at [3]. As for the > >>>>> previous rounds, my intention is to take a prefix of this series in= to > >>>>> 6.8, provided that it gets enough reviewing. > >>>>>=20 > >>>>> [1] https://lore.kernel.org/r/20230515173103.1017669-1-maz@kernel.o= rg > >>>>> [2] https://lore.kernel.org/r/20231120123721.851738-1-maz@kernel.org > >>>>> [3] https://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platfor= ms.git/log/?h=3Dkvm-arm64/nv-6.8-nv2-only > >>>>>=20 > >>>> While I was testing this with kvmtool for 5.16 I noted the following= on dmesg: > >>>>=20 > >>>> [ 803.014258] kvm [19040]: Unsupported guest sys_reg access at: 812= 9fa50 [600003c9] > >>>> { Op0( 3), Op1( 5), CRn( 1), CRm( 0), Op2( 2), func= _read }, > >>>>=20 > >>>> This is CPACR_EL12. > >>> CPACR_EL12 is redirected to VNCR[0x100]. It really shouldn't trap... > >>>=20 > >>>> Still need yet to debug. > >>> Can you disassemble the guest around the offending PC? > >>=20 > >> [ 1248.686350] kvm [7013]: Unsupported guest sys_reg access at: 812baa= 50 [600003c9] > >> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 { Op0( 3), Op1( 5), CRn( 1), CRm( 0), Op2( 2), func_r= ead }, > >>=20 > >> =C2=A012baa00:=C2=A0=C2=A0=C2=A0 14000008 =C2=A0=C2=A0=C2=A0 b=C2=A0= =C2=A0=C2=A0 0x12baa20 > >> =C2=A012baa04:=C2=A0=C2=A0=C2=A0 d000d501 =C2=A0=C2=A0=C2=A0 adrp=C2= =A0=C2=A0=C2=A0 x1, 0x2d5c000 > >> =C2=A012baa08:=C2=A0=C2=A0=C2=A0 91154021 =C2=A0=C2=A0=C2=A0 add=C2= =A0=C2=A0=C2=A0 x1, x1, #0x550 > >> =C2=A012baa0c:=C2=A0=C2=A0=C2=A0 f9400022 =C2=A0=C2=A0=C2=A0 ldr=C2= =A0=C2=A0=C2=A0 x2, [x1] > >> =C2=A012baa10:=C2=A0=C2=A0=C2=A0 f9400421 =C2=A0=C2=A0=C2=A0 ldr=C2= =A0=C2=A0=C2=A0 x1, [x1, #8] > >> =C2=A012baa14:=C2=A0=C2=A0=C2=A0 8a010042 =C2=A0=C2=A0=C2=A0 and=C2= =A0=C2=A0=C2=A0 x2, x2, x1 > >> =C2=A012baa18:=C2=A0=C2=A0=C2=A0 d3441c42 =C2=A0=C2=A0=C2=A0 ubfx=C2= =A0=C2=A0=C2=A0 x2, x2, #4, #4 > >> =C2=A012baa1c:=C2=A0=C2=A0=C2=A0 b4000082 =C2=A0=C2=A0=C2=A0 cbz=C2= =A0=C2=A0=C2=A0 x2, 0x12baa2c > >> =C2=A012baa20:=C2=A0=C2=A0=C2=A0 d2a175a0 =C2=A0=C2=A0=C2=A0 mov=C2= =A0=C2=A0=C2=A0 x0, #0xbad0000=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 // #195887104 > >> =C2=A012baa24:=C2=A0=C2=A0=C2=A0 f2994220 =C2=A0=C2=A0=C2=A0 movk=C2= =A0=C2=A0=C2=A0 x0, #0xca11 > >> =C2=A012baa28:=C2=A0=C2=A0=C2=A0 d69f03e0 =C2=A0=C2=A0=C2=A0 eret > >> =C2=A012baa2c:=C2=A0=C2=A0=C2=A0 d2c00080 =C2=A0=C2=A0=C2=A0 mov=C2= =A0=C2=A0=C2=A0 x0, #0x400000000=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 // #17179869184 > >> =C2=A012baa30:=C2=A0=C2=A0=C2=A0 f2b10000 =C2=A0=C2=A0=C2=A0 movk=C2= =A0=C2=A0=C2=A0 x0, #0x8800, lsl #16 > >> =C2=A012baa34:=C2=A0=C2=A0=C2=A0 f2800000 =C2=A0=C2=A0=C2=A0 movk=C2= =A0=C2=A0=C2=A0 x0, #0x0 > >> =C2=A012baa38:=C2=A0=C2=A0=C2=A0 d51c1100 =C2=A0=C2=A0=C2=A0 msr=C2= =A0=C2=A0=C2=A0 hcr_el2, x0 > >> =C2=A012baa3c:=C2=A0=C2=A0=C2=A0 d5033fdf =C2=A0=C2=A0=C2=A0 isb > >> =C2=A012baa40:=C2=A0=C2=A0=C2=A0 d53c4100 =C2=A0=C2=A0=C2=A0 mrs=C2= =A0=C2=A0=C2=A0 x0, sp_el1 > >> =C2=A012baa44:=C2=A0=C2=A0=C2=A0 9100001f =C2=A0=C2=A0=C2=A0 mov=C2= =A0=C2=A0=C2=A0 sp, x0 > >> =C2=A012baa48:=C2=A0=C2=A0=C2=A0 d538d080 =C2=A0=C2=A0=C2=A0 mrs=C2= =A0=C2=A0=C2=A0 x0, tpidr_el1 > >> =C2=A012baa4c:=C2=A0=C2=A0=C2=A0 d51cd040 =C2=A0=C2=A0=C2=A0 msr=C2= =A0=C2=A0=C2=A0 tpidr_el2, x0 > >> =C2=A012baa50:=C2=A0=C2=A0=C2=A0 d53d1040 =C2=A0=C2=A0=C2=A0 mrs=C2= =A0=C2=A0=C2=A0 x0, cpacr_el12 > >> =C2=A012baa54:=C2=A0=C2=A0=C2=A0 d5181040 =C2=A0=C2=A0=C2=A0 msr=C2= =A0=C2=A0=C2=A0 cpacr_el1, x0 > >> =C2=A012baa58:=C2=A0=C2=A0=C2=A0 d53dc000 =C2=A0=C2=A0=C2=A0 mrs=C2= =A0=C2=A0=C2=A0 x0, vbar_el12 > >> =C2=A012baa5c:=C2=A0=C2=A0=C2=A0 d518c000 =C2=A0=C2=A0=C2=A0 msr=C2= =A0=C2=A0=C2=A0 vbar_el1, x0 > >> =C2=A012baa60:=C2=A0=C2=A0=C2=A0 d53c1120 =C2=A0=C2=A0=C2=A0 mrs=C2= =A0=C2=A0=C2=A0 x0, mdcr_el2 > >> =C2=A012baa64:=C2=A0=C2=A0=C2=A0 9272f400 =C2=A0=C2=A0=C2=A0 and=C2= =A0=C2=A0=C2=A0 x0, x0, #0xffffffffffffcfff > >> =C2=A012baa68:=C2=A0=C2=A0=C2=A0 9266f400 =C2=A0=C2=A0=C2=A0 and=C2= =A0=C2=A0=C2=A0 x0, x0, #0xfffffffffcffffff > >> =C2=A012baa6c:=C2=A0=C2=A0=C2=A0 d51c1120 =C2=A0=C2=A0=C2=A0 msr=C2= =A0=C2=A0=C2=A0 mdcr_el2, x0 > >> =C2=A012baa70:=C2=A0=C2=A0=C2=A0 d53d2040 =C2=A0=C2=A0=C2=A0 mrs=C2= =A0=C2=A0=C2=A0 x0, tcr_el12 > >> =C2=A012baa74:=C2=A0=C2=A0=C2=A0 d5182040 =C2=A0=C2=A0=C2=A0 msr=C2= =A0=C2=A0=C2=A0 tcr_el1, x0 > >> =C2=A012baa78:=C2=A0=C2=A0=C2=A0 d53d2000 =C2=A0=C2=A0=C2=A0 mrs=C2= =A0=C2=A0=C2=A0 x0, ttbr0_el12 > >> =C2=A012baa7c:=C2=A0=C2=A0=C2=A0 d5182000 =C2=A0=C2=A0=C2=A0 msr=C2= =A0=C2=A0=C2=A0 ttbr0_el1, x0 > >> =C2=A012baa80:=C2=A0=C2=A0=C2=A0 d53d2020 =C2=A0=C2=A0=C2=A0 mrs=C2= =A0=C2=A0=C2=A0 x0, ttbr1_el12 > >> =C2=A012baa84:=C2=A0=C2=A0=C2=A0 d5182020 =C2=A0=C2=A0=C2=A0 msr=C2= =A0=C2=A0=C2=A0 ttbr1_el1, x0 > >> =C2=A012baa88:=C2=A0=C2=A0=C2=A0 d53da200 =C2=A0=C2=A0=C2=A0 mrs=C2= =A0=C2=A0=C2=A0 x0, mair_el12 > >> =C2=A012baa8c:=C2=A0=C2=A0=C2=A0 d518a200 =C2=A0=C2=A0=C2=A0 msr=C2= =A0=C2=A0=C2=A0 mair_el1, x0 > >> =C2=A012baa90:=C2=A0=C2=A0=C2=A0 d5380761 =C2=A0=C2=A0=C2=A0 mrs=C2= =A0=C2=A0=C2=A0 x1, s3_0_c0_c7_3 > >> =C2=A012baa94:=C2=A0=C2=A0=C2=A0 d3400c21 =C2=A0=C2=A0=C2=A0 ubfx=C2= =A0=C2=A0=C2=A0 x1, x1, #0, #4 > >> =C2=A012baa98:=C2=A0=C2=A0=C2=A0 b4000141 =C2=A0=C2=A0=C2=A0 cbz=C2= =A0=C2=A0=C2=A0 x1, 0x12baac0 > >> =C2=A012baa9c:=C2=A0=C2=A0=C2=A0 d53d2060 =C2=A0=C2=A0=C2=A0 mrs=C2= =A0=C2=A0=C2=A0 x0, s3_5_c2_c0_3 > >=20 > > OK, this is suspiciously close to the location Ganapatrao was having > > issues with. Are you running on the same hardware? > >=20 > > In any case, we should never take a trap for this access. Can you dump > > HCR_EL2 at the point where the guest traps (in switch.c)? > >=20 >=20 > I have dumped HCR_EL2 before entry to L1 in both V11 and V10. > on V10 HCR_EL2=3D0x2743c827c263f > on V11 HCR_EL2=3D0x27c3c827c263f >=20 > on V11 the function vcpu_el2_e2h_is_set(vcpu) is returning false > resulting in NV1 bit set along with NV and NV2. > AFAIK, For L1 to be in VHE, NV1 bit should be zero and NV=3DNV2=3D1. >=20 > I could boot L1 then L2, if I hack vcpu_el2_e2h_is_set to return true. > There could be a bug in V11 or E2H0 patchset resulting in > vcpu_el2_e2h_is_set() returning false? The E2H0 series should only force vcpu_el2_e2h_is_set() to return true, but not set it to false. Can you dump the *guest's* version of HCR_EL2 at this point? M. --=20 Without deviation from the norm, progress is not possible. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 424AAC61DF4 for ; Fri, 24 Nov 2023 10:19:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Subject:Cc:To:From:Message-ID:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=cwsJoO6kOtLwZEQYDBp5KaW2BGbTg9lcqcIVLDp5iss=; b=l70QLUvvPVg0CT zBzgEjV990mUhEAZI2aaNxpbmC2Cc0pazMQWcbESAywmkziQRE2jpzauBSEt1j3hNq1Q3d7PKGGcf wMasOTv4Eh05KRBfBlIrnLIjV0kBRgQuT14/rbXJsJ3Zs1sWeSjeeQN+JtLY+rN9F8//8/BVwJ3nF 9d3/PwRHm/dQiztLhfK/3pZGU3NSmzdyJftolZWMofQU9ALDJj8ToA4D+PeM8SPwuU3z/AzVyedKR WumTda46I/rIjAHj2G6PFCmqdEw9FMxgA+kZrrgK3Op0qkul5pQsS+11pEmX8ctkq1j4mxnlzcRRt 1Zg5zToJDvddFqNoTxxA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1r6THT-006oNS-2e; Fri, 24 Nov 2023 10:19:27 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1r6THQ-006oMs-2u for linux-arm-kernel@lists.infradead.org; Fri, 24 Nov 2023 10:19:26 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 6663462133; Fri, 24 Nov 2023 10:19:24 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1B394C433C8; Fri, 24 Nov 2023 10:19:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1700821164; bh=ezJrSp23KXb/hO6ks68IIX+fbvqWMNg+9nVoAxoKW8Q=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=szTiw9F2cGc2s71ZDdtpxoFF0EjWH2OY7N5AfnoWEE8zubiqo3Zn+JxLg7za7mO0p t1HV088SZOdi7zVKtTSwy2B/WJtDOXUyHQ4gLpzWm+9gi5B9Ih1rTTy1D0SdPOpwtT y0vjOpAuo08SS4pEyN2RL8Neyoj53swr+5je3F6etgitdJmzihA6SJntl/04QpCTQk RlzJ5ewHqXd/cpcRESrgv8emSxVUSEG34zj96iIOgcjK0L9G/lA7pBibiobDA1uryw pHTGB3TDAkEbxK+ulSe/wZFM5BC9vOZtQfYrdZYegvXd3tjM2KlCxsbXtwtU899BXC BHcf7Bh+ddkwg== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1r6THN-00G3Ie-Jc; Fri, 24 Nov 2023 10:19:21 +0000 Date: Fri, 24 Nov 2023 10:19:21 +0000 Message-ID: <868r6nzc5y.wl-maz@kernel.org> From: Marc Zyngier To: Ganapatrao Kulkarni Cc: Miguel Luis , "kvmarm@lists.linux.dev" , "kvm@vger.kernel.org" , "linux-arm-kernel@lists.infradead.org" , Alexandru Elisei , Andre Przywara , Chase Conklin , Christoffer Dall , Darren Hart , Jintack Lim , Russell King , James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu Subject: Re: [PATCH v11 00/43] KVM: arm64: Nested Virtualization support (FEAT_NV2 only) In-Reply-To: <134912e4-beed-4ab6-8ce1-33e69ec382b3@os.amperecomputing.com> References: <20231120131027.854038-1-maz@kernel.org> <86msv7ylnu.wl-maz@kernel.org> <05733774-4210-4097-9912-fb3aa8542fdd@oracle.com> <86a5r4zafh.wl-maz@kernel.org> <134912e4-beed-4ab6-8ce1-33e69ec382b3@os.amperecomputing.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/29.1 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: gankulkarni@os.amperecomputing.com, miguel.luis@oracle.com, kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, alexandru.elisei@arm.com, andre.przywara@arm.com, chase.conklin@arm.com, christoffer.dall@arm.com, darren@os.amperecomputing.com, jintack@cs.columbia.edu, rmk+kernel@armlinux.org.uk, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231124_021925_022906_6C5E2E4F X-CRM114-Status: GOOD ( 29.79 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gRnJpLCAyNCBOb3YgMjAyMyAwOTo1MDozMyArMDAwMCwKR2FuYXBhdHJhbyBLdWxrYXJuaSA8 Z2Fua3Vsa2FybmlAb3MuYW1wZXJlY29tcHV0aW5nLmNvbT4gd3JvdGU6Cj4gCj4gCj4gCj4gT24g MjMtMTEtMjAyMyAxMDoxNCBwbSwgTWFyYyBaeW5naWVyIHdyb3RlOgo+ID4gT24gVGh1LCAyMyBO b3YgMjAyMyAxNjoyMTo0OCArMDAwMCwKPiA+IE1pZ3VlbCBMdWlzIDxtaWd1ZWwubHVpc0BvcmFj bGUuY29tPiB3cm90ZToKPiA+PiAKPiA+PiBIaSBNYXJjLAo+ID4+IAo+ID4+IE9uIDIxLzExLzIw MjMgMTg6MDIsIE1hcmMgWnluZ2llciB3cm90ZToKPiA+Pj4gT24gVHVlLCAyMSBOb3YgMjAyMyAx Njo0OTo1MiArMDAwMCwKPiA+Pj4gTWlndWVsIEx1aXMgPG1pZ3VlbC5sdWlzQG9yYWNsZS5jb20+ IHdyb3RlOgo+ID4+Pj4gSGkgTWFyYywKPiA+Pj4+IAo+ID4+Pj4+IE9uIDIwIE5vdiAyMDIzLCBh dCAxMjowOSwgTWFyYyBaeW5naWVyIDxtYXpAa2VybmVsLm9yZz4gd3JvdGU6Cj4gPj4+Pj4gCj4g Pj4+Pj4gVGhpcyBpcyB0aGUgNXRoIGRyb3Agb2YgTlYgc3VwcG9ydCBvbiBhcm02NCBmb3IgdGhp cyB5ZWFyLCBhbmQgbW9zdAo+ID4+Pj4+IHByb2JhYmx5IHRoZSBsYXN0IG9uZSBmb3IgdGhpcyBz aWRlIG9mIENocmlzdG1hcy4KPiA+Pj4+PiAKPiA+Pj4+PiBGb3IgdGhlIHByZXZpb3VzIGVwaXNv ZGVzLCBzZWUgWzFdLgo+ID4+Pj4+IAo+ID4+Pj4+IFdoYXQncyBjaGFuZ2VkOgo+ID4+Pj4+IAo+ ID4+Pj4+IC0gRHJvcCBzdXBwb3J0IGZvciB0aGUgb3JpZ2luYWwgRkVBVF9OVi4gTm8gZXhpc3Rp bmcgaGFyZHdhcmUgc3VwcG9ydHMKPiA+Pj4+PiAgIGl0IHdpdGhvdXQgRkVBVF9OVjIsIGFuZCB0 aGUgYXJjaGl0ZWN0dXJlIGlzIGRlcHJlY2F0aW5nIHRoZSBmb3JtZXIKPiA+Pj4+PiAgIGVudGly ZWx5LiBUaGlzIHJlc3VsdHMgaW4gZmV3ZXIgcGF0Y2hlcywgYW5kIGEgc2xpZ2h0bHkgc2ltcGxl cgo+ID4+Pj4+ICAgbW9kZWwgb3ZlcmFsbC4KPiA+Pj4+PiAKPiA+Pj4+PiAtIFJlb3JnYW5pc2Ug dGhlIHNlcmllcyB0byBtYWtlIGl0IGEgYml0IG1vcmUgbG9naWNhbCBub3cgdGhhdCBGRUFUX05W Cj4gPj4+Pj4gICBpcyBnb25lLgo+ID4+Pj4+IAo+ID4+Pj4+IC0gQXBwbHkgdGhlIE5WIGlkcmVn IHJlc3RyaWN0aW9ucyBvbiBWTSBmaXJzdCBydW4gcmF0aGVyIHRoYW4gb24gZWFjaAo+ID4+Pj4+ ICAgYWNjZXNzLgo+ID4+Pj4+IAo+ID4+Pj4+IC0gTWFrZSB0aGUgbmVzdGVkIHZnaWMgc2hhZG93 IENQVSBpbnRlcmZhY2UgYSBwZXItQ1BVIHN0cnVjdHVyZSByYXRoZXIKPiA+Pj4+PiAgIHRoYW4g cGVyLXZjcHUuCj4gPj4+Pj4gCj4gPj4+Pj4gLSBGaXggdGhlIEVMMCB0aW1lciBmYXN0cGF0aAo+ ID4+Pj4+IAo+ID4+Pj4+IC0gV29yayBhcm91bmQgdGhlIGFyY2hpdGVjdHVyZSBkZWZpY2llbmNp ZXMgd2hlbiB0cmFwcGluZyBXRkkgZnJvbSBhCj4gPj4+Pj4gICBMMiBndWVzdC4KPiA+Pj4+PiAK PiA+Pj4+PiAtIEZpeCBzYW1wbGluZyBvZiBuZXN0ZWQgdmdpYyBzdGF0ZSAoTUlTUiwgRUxSU1Is IEVJU1IpCj4gPj4+Pj4gCj4gPj4+Pj4gLSBEcm9wIHRoZSBwYXRjaGVzIHRoYXQgaGF2ZSBhbHJl YWR5IGJlZW4gbWVyZ2VkIChOViB0cmFwIGZvcndhcmRpbmcsCj4gPj4+Pj4gICBwZXItTU1VIFZU Q1IpCj4gPj4+Pj4gCj4gPj4+Pj4gLSBSZWJhc2VkIG9uIHRvcCBvZiA2LjctcmMyICsgdGhlIEZF QVRfRTJIMCBzdXBwb3J0IFsyXS4KPiA+Pj4+PiAKPiA+Pj4+PiBUaGUgYnJhbmNoIGNvbnRhaW5p bmcgdGhlc2UgcGF0Y2hlcyAoYW5kIG1vcmUpIGlzIGF0IFszXS4gQXMgZm9yIHRoZQo+ID4+Pj4+ IHByZXZpb3VzIHJvdW5kcywgbXkgaW50ZW50aW9uIGlzIHRvIHRha2UgYSBwcmVmaXggb2YgdGhp cyBzZXJpZXMgaW50bwo+ID4+Pj4+IDYuOCwgcHJvdmlkZWQgdGhhdCBpdCBnZXRzIGVub3VnaCBy ZXZpZXdpbmcuCj4gPj4+Pj4gCj4gPj4+Pj4gWzFdIGh0dHBzOi8vbG9yZS5rZXJuZWwub3JnL3Iv MjAyMzA1MTUxNzMxMDMuMTAxNzY2OS0xLW1hekBrZXJuZWwub3JnCj4gPj4+Pj4gWzJdIGh0dHBz Oi8vbG9yZS5rZXJuZWwub3JnL3IvMjAyMzExMjAxMjM3MjEuODUxNzM4LTEtbWF6QGtlcm5lbC5v cmcKPiA+Pj4+PiBbM10gaHR0cHM6Ly9naXQua2VybmVsLm9yZy9wdWIvc2NtL2xpbnV4L2tlcm5l bC9naXQvbWF6L2FybS1wbGF0Zm9ybXMuZ2l0L2xvZy8/aD1rdm0tYXJtNjQvbnYtNi44LW52Mi1v bmx5Cj4gPj4+Pj4gCj4gPj4+PiBXaGlsZSBJIHdhcyB0ZXN0aW5nIHRoaXMgd2l0aCBrdm10b29s IGZvciA1LjE2IEkgbm90ZWQgdGhlIGZvbGxvd2luZyBvbiBkbWVzZzoKPiA+Pj4+IAo+ID4+Pj4g WyAgODAzLjAxNDI1OF0ga3ZtIFsxOTA0MF06IFVuc3VwcG9ydGVkIGd1ZXN0IHN5c19yZWcgYWNj ZXNzIGF0OiA4MTI5ZmE1MCBbNjAwMDAzYzldCj4gPj4+PiAgICAgICAgICAgICAgICAgIHsgT3Aw KCAzKSwgT3AxKCA1KSwgQ1JuKCAxKSwgQ1JtKCAwKSwgT3AyKCAyKSwgZnVuY19yZWFkIH0sCj4g Pj4+PiAKPiA+Pj4+IFRoaXMgaXMgQ1BBQ1JfRUwxMi4KPiA+Pj4gQ1BBQ1JfRUwxMiBpcyByZWRp cmVjdGVkIHRvIFZOQ1JbMHgxMDBdLiBJdCByZWFsbHkgc2hvdWxkbid0IHRyYXAuLi4KPiA+Pj4g Cj4gPj4+PiBTdGlsbCBuZWVkIHlldCB0byBkZWJ1Zy4KPiA+Pj4gQ2FuIHlvdSBkaXNhc3NlbWJs ZSB0aGUgZ3Vlc3QgYXJvdW5kIHRoZSBvZmZlbmRpbmcgUEM/Cj4gPj4gCj4gPj4gWyAxMjQ4LjY4 NjM1MF0ga3ZtIFs3MDEzXTogVW5zdXBwb3J0ZWQgZ3Vlc3Qgc3lzX3JlZyBhY2Nlc3MgYXQ6IDgx MmJhYTUwIFs2MDAwMDNjOV0KPiA+PiAgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHsg T3AwKCAzKSwgT3AxKCA1KSwgQ1JuKCAxKSwgQ1JtKCAwKSwgT3AyKCAyKSwgZnVuY19yZWFkIH0s Cj4gPj4gCj4gPj4gIMKgMTJiYWEwMDrCoMKgwqAgMTQwMDAwMDggwqDCoMKgIGLCoMKgwqAgMHgx MmJhYTIwCj4gPj4gIMKgMTJiYWEwNDrCoMKgwqAgZDAwMGQ1MDEgwqDCoMKgIGFkcnDCoMKgwqAg eDEsIDB4MmQ1YzAwMAo+ID4+ICDCoDEyYmFhMDg6wqDCoMKgIDkxMTU0MDIxIMKgwqDCoCBhZGTC oMKgwqAgeDEsIHgxLCAjMHg1NTAKPiA+PiAgwqAxMmJhYTBjOsKgwqDCoCBmOTQwMDAyMiDCoMKg wqAgbGRywqDCoMKgIHgyLCBbeDFdCj4gPj4gIMKgMTJiYWExMDrCoMKgwqAgZjk0MDA0MjEgwqDC oMKgIGxkcsKgwqDCoCB4MSwgW3gxLCAjOF0KPiA+PiAgwqAxMmJhYTE0OsKgwqDCoCA4YTAxMDA0 MiDCoMKgwqAgYW5kwqDCoMKgIHgyLCB4MiwgeDEKPiA+PiAgwqAxMmJhYTE4OsKgwqDCoCBkMzQ0 MWM0MiDCoMKgwqAgdWJmeMKgwqDCoCB4MiwgeDIsICM0LCAjNAo+ID4+ICDCoDEyYmFhMWM6wqDC oMKgIGI0MDAwMDgyIMKgwqDCoCBjYnrCoMKgwqAgeDIsIDB4MTJiYWEyYwo+ID4+ICDCoDEyYmFh MjA6wqDCoMKgIGQyYTE3NWEwIMKgwqDCoCBtb3bCoMKgwqAgeDAsICMweGJhZDAwMDDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqAgwqDCoMKgIC8vICMxOTU4ODcxMDQKPiA+PiAgwqAxMmJhYTI0OsKg wqDCoCBmMjk5NDIyMCDCoMKgwqAgbW92a8KgwqDCoCB4MCwgIzB4Y2ExMQo+ID4+ICDCoDEyYmFh Mjg6wqDCoMKgIGQ2OWYwM2UwIMKgwqDCoCBlcmV0Cj4gPj4gIMKgMTJiYWEyYzrCoMKgwqAgZDJj MDAwODAgwqDCoMKgIG1vdsKgwqDCoCB4MCwgIzB4NDAwMDAwMDAwwqDCoMKgwqDCoMKgwqDCoMKg wqAgwqDCoMKgIC8vICMxNzE3OTg2OTE4NAo+ID4+ICDCoDEyYmFhMzA6wqDCoMKgIGYyYjEwMDAw IMKgwqDCoCBtb3ZrwqDCoMKgIHgwLCAjMHg4ODAwLCBsc2wgIzE2Cj4gPj4gIMKgMTJiYWEzNDrC oMKgwqAgZjI4MDAwMDAgwqDCoMKgIG1vdmvCoMKgwqAgeDAsICMweDAKPiA+PiAgwqAxMmJhYTM4 OsKgwqDCoCBkNTFjMTEwMCDCoMKgwqAgbXNywqDCoMKgIGhjcl9lbDIsIHgwCj4gPj4gIMKgMTJi YWEzYzrCoMKgwqAgZDUwMzNmZGYgwqDCoMKgIGlzYgo+ID4+ICDCoDEyYmFhNDA6wqDCoMKgIGQ1 M2M0MTAwIMKgwqDCoCBtcnPCoMKgwqAgeDAsIHNwX2VsMQo+ID4+ICDCoDEyYmFhNDQ6wqDCoMKg IDkxMDAwMDFmIMKgwqDCoCBtb3bCoMKgwqAgc3AsIHgwCj4gPj4gIMKgMTJiYWE0ODrCoMKgwqAg ZDUzOGQwODAgwqDCoMKgIG1yc8KgwqDCoCB4MCwgdHBpZHJfZWwxCj4gPj4gIMKgMTJiYWE0YzrC oMKgwqAgZDUxY2QwNDAgwqDCoMKgIG1zcsKgwqDCoCB0cGlkcl9lbDIsIHgwCj4gPj4gIMKgMTJi YWE1MDrCoMKgwqAgZDUzZDEwNDAgwqDCoMKgIG1yc8KgwqDCoCB4MCwgY3BhY3JfZWwxMgo+ID4+ ICDCoDEyYmFhNTQ6wqDCoMKgIGQ1MTgxMDQwIMKgwqDCoCBtc3LCoMKgwqAgY3BhY3JfZWwxLCB4 MAo+ID4+ICDCoDEyYmFhNTg6wqDCoMKgIGQ1M2RjMDAwIMKgwqDCoCBtcnPCoMKgwqAgeDAsIHZi YXJfZWwxMgo+ID4+ICDCoDEyYmFhNWM6wqDCoMKgIGQ1MThjMDAwIMKgwqDCoCBtc3LCoMKgwqAg dmJhcl9lbDEsIHgwCj4gPj4gIMKgMTJiYWE2MDrCoMKgwqAgZDUzYzExMjAgwqDCoMKgIG1yc8Kg wqDCoCB4MCwgbWRjcl9lbDIKPiA+PiAgwqAxMmJhYTY0OsKgwqDCoCA5MjcyZjQwMCDCoMKgwqAg YW5kwqDCoMKgIHgwLCB4MCwgIzB4ZmZmZmZmZmZmZmZmY2ZmZgo+ID4+ICDCoDEyYmFhNjg6wqDC oMKgIDkyNjZmNDAwIMKgwqDCoCBhbmTCoMKgwqAgeDAsIHgwLCAjMHhmZmZmZmZmZmZjZmZmZmZm Cj4gPj4gIMKgMTJiYWE2YzrCoMKgwqAgZDUxYzExMjAgwqDCoMKgIG1zcsKgwqDCoCBtZGNyX2Vs MiwgeDAKPiA+PiAgwqAxMmJhYTcwOsKgwqDCoCBkNTNkMjA0MCDCoMKgwqAgbXJzwqDCoMKgIHgw LCB0Y3JfZWwxMgo+ID4+ICDCoDEyYmFhNzQ6wqDCoMKgIGQ1MTgyMDQwIMKgwqDCoCBtc3LCoMKg wqAgdGNyX2VsMSwgeDAKPiA+PiAgwqAxMmJhYTc4OsKgwqDCoCBkNTNkMjAwMCDCoMKgwqAgbXJz wqDCoMKgIHgwLCB0dGJyMF9lbDEyCj4gPj4gIMKgMTJiYWE3YzrCoMKgwqAgZDUxODIwMDAgwqDC oMKgIG1zcsKgwqDCoCB0dGJyMF9lbDEsIHgwCj4gPj4gIMKgMTJiYWE4MDrCoMKgwqAgZDUzZDIw MjAgwqDCoMKgIG1yc8KgwqDCoCB4MCwgdHRicjFfZWwxMgo+ID4+ICDCoDEyYmFhODQ6wqDCoMKg IGQ1MTgyMDIwIMKgwqDCoCBtc3LCoMKgwqAgdHRicjFfZWwxLCB4MAo+ID4+ICDCoDEyYmFhODg6 wqDCoMKgIGQ1M2RhMjAwIMKgwqDCoCBtcnPCoMKgwqAgeDAsIG1haXJfZWwxMgo+ID4+ICDCoDEy YmFhOGM6wqDCoMKgIGQ1MThhMjAwIMKgwqDCoCBtc3LCoMKgwqAgbWFpcl9lbDEsIHgwCj4gPj4g IMKgMTJiYWE5MDrCoMKgwqAgZDUzODA3NjEgwqDCoMKgIG1yc8KgwqDCoCB4MSwgczNfMF9jMF9j N18zCj4gPj4gIMKgMTJiYWE5NDrCoMKgwqAgZDM0MDBjMjEgwqDCoMKgIHViZnjCoMKgwqAgeDEs IHgxLCAjMCwgIzQKPiA+PiAgwqAxMmJhYTk4OsKgwqDCoCBiNDAwMDE0MSDCoMKgwqAgY2J6wqDC oMKgIHgxLCAweDEyYmFhYzAKPiA+PiAgwqAxMmJhYTljOsKgwqDCoCBkNTNkMjA2MCDCoMKgwqAg bXJzwqDCoMKgIHgwLCBzM181X2MyX2MwXzMKPiA+IAo+ID4gT0ssIHRoaXMgaXMgc3VzcGljaW91 c2x5IGNsb3NlIHRvIHRoZSBsb2NhdGlvbiBHYW5hcGF0cmFvIHdhcyBoYXZpbmcKPiA+IGlzc3Vl cyB3aXRoLiBBcmUgeW91IHJ1bm5pbmcgb24gdGhlIHNhbWUgaGFyZHdhcmU/Cj4gPiAKPiA+IElu IGFueSBjYXNlLCB3ZSBzaG91bGQgbmV2ZXIgdGFrZSBhIHRyYXAgZm9yIHRoaXMgYWNjZXNzLiBD YW4geW91IGR1bXAKPiA+IEhDUl9FTDIgYXQgdGhlIHBvaW50IHdoZXJlIHRoZSBndWVzdCB0cmFw cyAoaW4gc3dpdGNoLmMpPwo+ID4gCj4gCj4gSSBoYXZlIGR1bXBlZCBIQ1JfRUwyIGJlZm9yZSBl bnRyeSB0byBMMSBpbiBib3RoIFYxMSBhbmQgVjEwLgo+IG9uIFYxMCBIQ1JfRUwyPTB4Mjc0M2M4 MjdjMjYzZgo+IG9uIFYxMSBIQ1JfRUwyPTB4MjdjM2M4MjdjMjYzZgo+IAo+IG9uIFYxMSB0aGUg ZnVuY3Rpb24gdmNwdV9lbDJfZTJoX2lzX3NldCh2Y3B1KSBpcyByZXR1cm5pbmcgZmFsc2UKPiBy ZXN1bHRpbmcgaW4gTlYxIGJpdCBzZXQgYWxvbmcgd2l0aCBOViBhbmQgTlYyLgo+IEFGQUlLLCBG b3IgTDEgdG8gYmUgaW4gVkhFLCBOVjEgYml0IHNob3VsZCBiZSB6ZXJvIGFuZCBOVj1OVjI9MS4K PiAKPiBJIGNvdWxkIGJvb3QgTDEgdGhlbiBMMiwgaWYgSSBoYWNrIHZjcHVfZWwyX2UyaF9pc19z ZXQgdG8gcmV0dXJuIHRydWUuCj4gVGhlcmUgY291bGQgYmUgYSBidWcgaW4gVjExIG9yIEUySDAg cGF0Y2hzZXQgcmVzdWx0aW5nIGluCj4gdmNwdV9lbDJfZTJoX2lzX3NldCgpIHJldHVybmluZyBm YWxzZT8KClRoZSBFMkgwIHNlcmllcyBzaG91bGQgb25seSBmb3JjZSB2Y3B1X2VsMl9lMmhfaXNf c2V0KCkgdG8gcmV0dXJuCnRydWUsIGJ1dCBub3Qgc2V0IGl0IHRvIGZhbHNlLiBDYW4geW91IGR1 bXAgdGhlICpndWVzdCdzKiB2ZXJzaW9uIG9mCkhDUl9FTDIgYXQgdGhpcyBwb2ludD8KCglNLgoK LS0gCldpdGhvdXQgZGV2aWF0aW9uIGZyb20gdGhlIG5vcm0sIHByb2dyZXNzIGlzIG5vdCBwb3Nz aWJsZS4KCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCmxp bnV4LWFybS1rZXJuZWwgbWFpbGluZyBsaXN0CmxpbnV4LWFybS1rZXJuZWxAbGlzdHMuaW5mcmFk ZWFkLm9yZwpodHRwOi8vbGlzdHMuaW5mcmFkZWFkLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2xpbnV4 LWFybS1rZXJuZWwK