From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2A50AC433F5 for ; Thu, 27 Jan 2022 13:05:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Subject:Cc:To:From:Message-ID:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=22ZdT6su0fYLwm8of0cduB6jmghwtdakUfMNn7DpNuI=; b=jOWYGLTszrWJtd zmknOb2CiAHf0qv6wS1VcnBFd+52QgVpnT23rl3qV9wueyi2TvBTiWXsZdglyMO/aHRCC6m9fE371 x0bgzFsbLgpCilNqMnHE/DVTeOYrkbwFb1PP6wpwFfJ+BApKAifCZ0gTRQWRgViWJwy8UL3c3ZANK 7WjIrzg7eY1pQ2SUa/L/+hkL+iNPMtw5uQVeZzINrGGvpb5in4iCt5Y6TRbWTBdFW3UngpsmkMZnB ZuRZv3YC38FKEO0npXmHFD8X0ueQJVsyEreZu/LOs9iyXFl5Mo6AMP3PU9brcjDEsEZ3Y1/m2dFek V7jhKULlOmJag5iLKw6g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nD4Rf-00Focu-Sm; Thu, 27 Jan 2022 13:04:12 +0000 Received: from ams.source.kernel.org ([2604:1380:4601:e00::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nD4Rb-00Foc1-Se for linux-arm-kernel@lists.infradead.org; Thu, 27 Jan 2022 13:04:09 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 57720B82236; Thu, 27 Jan 2022 13:04:06 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E8CCEC340E4; Thu, 27 Jan 2022 13:04:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1643288645; bh=qMZ0svqK5htZaSgXJT3ca1NFnVOJRjs9aPXrpCGNpFs=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=K6HFDmz6fZyaYO5TfEDPf3JzSHva9TrXwR663EBPsuJyagrEruThk7SHKj2XArCGd 7OVA6Bb/YnYTLV9pH3qVOkFlMfUge1J7DZ9wLYAnHGl20yG06LmV8Ry0zRKqriIORg 3M/VQHKkgWiqJhizgr2s8kx/6iLqcrWVFqGAdz7FOLrWz4v165lWgHWj7PYKry/MZ9 xDHbCHahuNQu/91zdAINUtFyZeiKqSxng9vDf6O2CVjAq3ow7IK9/qJfpznbkW8KCx 0OfuoeWeMe5GoVtKSnP4acDnS5QyqyfQGL6s0MeM1y5y33KESwlT8UY8lVe0Qrsn4Z qBzJOMBCKo70g== Received: from sofa.misterjones.org ([185.219.108.64] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1nD4RW-003Vpx-Uc; Thu, 27 Jan 2022 13:04:03 +0000 Date: Thu, 27 Jan 2022 13:04:02 +0000 Message-ID: <87v8y55nzx.wl-maz@kernel.org> From: Marc Zyngier To: Ganapatrao Kulkarni Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, Andre Przywara , Christoffer Dall , Jintack Lim , Haibo Xu , James Morse , Suzuki K Poulose , Alexandru Elisei , kernel-team@android.com Subject: Re: [PATCH v5 36/69] KVM: arm64: nv: Filter out unsupported features from ID regs In-Reply-To: <300f7a61-acd1-21bd-d36d-1532d2cecf44@os.amperecomputing.com> References: <20211129200150.351436-1-maz@kernel.org> <20211129200150.351436-37-maz@kernel.org> <87h7b3wqe9.wl-maz@kernel.org> <87zgouuxvy.wl-maz@kernel.org> <300f7a61-acd1-21bd-d36d-1532d2cecf44@os.amperecomputing.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: gankulkarni@os.amperecomputing.com, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, andre.przywara@arm.com, christoffer.dall@arm.com, jintack@cs.columbia.edu, haibo.xu@linaro.org, james.morse@arm.com, suzuki.poulose@arm.com, alexandru.elisei@arm.com, kernel-team@android.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220127_050408_232060_2C4B301E X-CRM114-Status: GOOD ( 46.46 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, 21 Jan 2022 11:33:30 +0000, Ganapatrao Kulkarni wrote: > > Hi Marc, > > On 21-12-2021 02:40 pm, Marc Zyngier wrote: > > On Tue, 21 Dec 2021 06:03:49 +0000, > > Ganapatrao Kulkarni wrote: > >> > >> > >> > >> On 20-12-2021 03:26 pm, Marc Zyngier wrote: > >>> On Mon, 20 Dec 2021 07:26:50 +0000, > >>> Ganapatrao Kulkarni wrote: > >>>> > >>>> > >>>> Hi Marc, > >>>> > >>>> On 30-11-2021 01:31 am, Marc Zyngier wrote: > >>>>> As there is a number of features that we either can't support, > >>>>> or don't want to support right away with NV, let's add some > >>>>> basic filtering so that we don't advertize silly things to the > >>>>> EL2 guest. > >>>>> > >>>>> Whilst we are at it, avertize ARMv8.4-TTL as well as ARMv8.5-GTG. > >>>>> > >>>>> Signed-off-by: Marc Zyngier > >>>>> --- > >>>>> arch/arm64/include/asm/kvm_nested.h | 6 ++ > >>>>> arch/arm64/kvm/nested.c | 152 ++++++++++++++++++++++++++++ > >>>>> arch/arm64/kvm/sys_regs.c | 4 +- > >>>>> arch/arm64/kvm/sys_regs.h | 2 + > >>>>> 4 files changed, 163 insertions(+), 1 deletion(-) > >>>>> > >>>>> diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h > >>>>> index 07c15f51cf86..026ddaad972c 100644 > >>>>> --- a/arch/arm64/include/asm/kvm_nested.h > >>>>> +++ b/arch/arm64/include/asm/kvm_nested.h > >>>>> @@ -67,4 +67,10 @@ extern bool __forward_traps(struct kvm_vcpu *vcpu, unsigned int reg, > >>>>> extern bool forward_traps(struct kvm_vcpu *vcpu, u64 control_bit); > >>>>> extern bool forward_nv_traps(struct kvm_vcpu *vcpu); > >>>>> +struct sys_reg_params; > >>>>> +struct sys_reg_desc; > >>>>> + > >>>>> +void access_nested_id_reg(struct kvm_vcpu *v, struct sys_reg_params *p, > >>>>> + const struct sys_reg_desc *r); > >>>>> + > >>>>> #endif /* __ARM64_KVM_NESTED_H */ > >>>>> diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c > >>>>> index 42a96c8d2adc..19b674983e13 100644 > >>>>> --- a/arch/arm64/kvm/nested.c > >>>>> +++ b/arch/arm64/kvm/nested.c > >>>>> @@ -20,6 +20,10 @@ > >>>>> #include > >>>>> #include > >>>>> +#include > >>>>> +#include > >>>>> + > >>>>> +#include "sys_regs.h" > >>>>> /* > >>>>> * Inject wfx to the virtual EL2 if this is not from the virtual EL2 and > >>>>> @@ -38,3 +42,151 @@ int handle_wfx_nested(struct kvm_vcpu *vcpu, bool is_wfe) > >>>>> return -EINVAL; > >>>>> } > >>>>> + > >>>>> +/* > >>>>> + * Our emulated CPU doesn't support all the possible features. For the > >>>>> + * sake of simplicity (and probably mental sanity), wipe out a number > >>>>> + * of feature bits we don't intend to support for the time being. > >>>>> + * This list should get updated as new features get added to the NV > >>>>> + * support, and new extension to the architecture. > >>>>> + */ > >>>>> +void access_nested_id_reg(struct kvm_vcpu *v, struct sys_reg_params *p, > >>>>> + const struct sys_reg_desc *r) > >>>>> +{ > >>>>> + u32 id = sys_reg((u32)r->Op0, (u32)r->Op1, > >>>>> + (u32)r->CRn, (u32)r->CRm, (u32)r->Op2); > >>>>> + u64 val, tmp; > >>>>> + > >>>>> + if (!nested_virt_in_use(v)) > >>>>> + return; > >>>>> + > >>>>> + val = p->regval; > >>>>> + > >>>>> + switch (id) { > >>>>> + case SYS_ID_AA64ISAR0_EL1: > >>>>> + /* Support everything but O.S. and Range TLBIs */ > >>>>> + val &= ~(FEATURE(ID_AA64ISAR0_TLB) | > >>>>> + GENMASK_ULL(27, 24) | > >>>>> + GENMASK_ULL(3, 0)); > >>>>> + break; > >>>>> + > >>>>> + case SYS_ID_AA64ISAR1_EL1: > >>>>> + /* Support everything but PtrAuth and Spec Invalidation */ > >>>>> + val &= ~(GENMASK_ULL(63, 56) | > >>>>> + FEATURE(ID_AA64ISAR1_SPECRES) | > >>>>> + FEATURE(ID_AA64ISAR1_GPI) | > >>>>> + FEATURE(ID_AA64ISAR1_GPA) | > >>>>> + FEATURE(ID_AA64ISAR1_API) | > >>>>> + FEATURE(ID_AA64ISAR1_APA)); > >>>>> + break; > >>>>> + > >>>>> + case SYS_ID_AA64PFR0_EL1: > >>>>> + /* No AMU, MPAM, S-EL2, RAS or SVE */ > >>>>> + val &= ~(GENMASK_ULL(55, 52) | > >>>>> + FEATURE(ID_AA64PFR0_AMU) | > >>>>> + FEATURE(ID_AA64PFR0_MPAM) | > >>>>> + FEATURE(ID_AA64PFR0_SEL2) | > >>>>> + FEATURE(ID_AA64PFR0_RAS) | > >>>>> + FEATURE(ID_AA64PFR0_SVE) | > >>>>> + FEATURE(ID_AA64PFR0_EL3) | > >>>>> + FEATURE(ID_AA64PFR0_EL2)); > >>>>> + /* 64bit EL2/EL3 only */ > >>>>> + val |= FIELD_PREP(FEATURE(ID_AA64PFR0_EL2), 0b0001); > >>>>> + val |= FIELD_PREP(FEATURE(ID_AA64PFR0_EL3), 0b0001); > >>>>> + break; > >>>>> + > >>>>> + case SYS_ID_AA64PFR1_EL1: > >>>>> + /* Only support SSBS */ > >>>>> + val &= FEATURE(ID_AA64PFR1_SSBS); > >>>>> + break; > >>>>> + > >>>>> + case SYS_ID_AA64MMFR0_EL1: > >>>>> + /* Hide ECV, FGT, ExS, Secure Memory */ > >>>>> + val &= ~(GENMASK_ULL(63, 43) | > >>>>> + FEATURE(ID_AA64MMFR0_TGRAN4_2) | > >>>>> + FEATURE(ID_AA64MMFR0_TGRAN16_2) | > >>>>> + FEATURE(ID_AA64MMFR0_TGRAN64_2) | > >>>>> + FEATURE(ID_AA64MMFR0_SNSMEM)); > >>>>> + > >>>>> + /* Disallow unsupported S2 page sizes */ > >>>>> + switch (PAGE_SIZE) { > >>>>> + case SZ_64K: > >>>>> + val |= FIELD_PREP(FEATURE(ID_AA64MMFR0_TGRAN16_2), 0b0001); > >>>>> + fallthrough; > >>>>> + case SZ_16K: > >>>>> + val |= FIELD_PREP(FEATURE(ID_AA64MMFR0_TGRAN4_2), 0b0001); > >>>>> + fallthrough; > >>>>> + case SZ_4K: > >>>>> + /* Support everything */ > >>>>> + break; > >>>>> + } > >>>> > >>>> It seems to me that Host hypervisor(L0) has to boot with 4KB page size > >>>> to support all (4, 16 and 64KB) page sizes at L1, any specific reason > >>>> for this restriction? > >>> > >>> Well, yes. > >>> > >>> If you have a L0 that has booted with (let's say) 64kB page size, how > >>> do you provide S2 mappings with 4kB granularity so that you can > >>> implement the permissions that a L1 guest hypervisor can impose on its > >>> own guest, given that KVM currently mandates S1 and S2 to use the same > >>> page sizes? > >>> > >>> You can't. That's why we tell the guest hypervisor how much we > >>> support, and the guest hypervisor can decide to go ahead or not > >>> depending on what it does. > >>> > >>> If one day we can support S2 mappings that are smaller than the host > >>> page sizes, then we'll be able to allow to advertise all page sizes. > >>> But I wouldn't hold my breath for this to happen. > >> > >> Thanks for the detailed explanation!. > >> Can we put one line comment that explains why this manipulation? > >> It would be helpful to see a comment like S2 PAGE_SIZE should be > >> at-least the size of Host PAGE_SIZE? > > > > Can do, but we need to get the terminology straight, because this is > > very quickly becoming confusing. Something like: > > > > diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c > > index 7c9dd1edf011..d35a947f5679 100644 > > --- a/arch/arm64/kvm/nested.c > > +++ b/arch/arm64/kvm/nested.c > > @@ -850,7 +850,12 @@ void access_nested_id_reg(struct kvm_vcpu *v, struct sys_reg_params *p, > > /* Support everything */ > > break; > > } > > - /* Advertize supported S2 page sizes */ > > + /* > > + * Since we can't support a guest S2 page size smaller than > > + * the host's own page size (due to KVM only populating its > > + * own S2 using the kernel's page size), advertise the > > + * limitation using FEAT_GTG. > > + */ > > I have tried booting L0 with 4K page-size and L1 with 64K and with > this config the L2/NestedVM boot hangs. I have tried L2 with > page-sizes 4K and 64K(though S1 page size of L2 should not matter?). S1 shouldn't matter, but if that's what you are seeing, there is obviously an issue trying to satisfy a translation fault in this configuration, and we get stuck. Could you please add some tracing and work out whether this is the case? I bet this is an issue trying to combine the L1 S2 translation (which will be 64kB aligned) with the faulting IPA address, but I can't immediately pinpoint the bug. Thanks, M. -- Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel