From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.0 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE, SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A4178C2D0C6 for ; Sat, 28 Dec 2019 11:43:08 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 65FE8207E0 for ; Sat, 28 Dec 2019 11:43:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="MyOMSEvG" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 65FE8207E0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Subject:To:From:Message-ID:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=gzwx+WDguZ7ziLETcmeMj+6JcIV58dFF6pizVBXNPSI=; b=MyOMSEvGzdrD05 O8IphHAKPckTOaIKHpLMwbj0fZOmmNcQgOPUAwO+ArardmRC/FknzmYIkvOrpMCx6V9pGwUQrqTev CX3aY4Wq/iwdTVPxkThkGfzye6VrHGZQqZ45mXcy0jh09I6JQgUZ/K8lufX6B2cCNoTXFyiaoOEzl PtrDj2cY0DLSGrPRig0N8HeRwxFQdn6Gn6pdSTLDe9Bf8uhfWotZE2N28tHd9TASCERd5wCSA3duv oulDyye+5hmIujqAAvX+1ApiOOMPAbUdQPoOth7MfhKRa/klc6jxC9mg1aA5PZdoA7qi2cUW6oqht 1+HFPiVo5mVF8J6laM0Q==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1ilAUt-0001iH-Av; Sat, 28 Dec 2019 11:43:07 +0000 Received: from disco-boy.misterjones.org ([51.254.78.96]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1ilAUq-0001hc-DE for linux-arm-kernel@lists.infradead.org; Sat, 28 Dec 2019 11:43:05 +0000 Received: from 78.163-31-62.static.virginmediabusiness.co.uk ([62.31.163.78] helo=big-swifty.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1ilALz-0004YB-Ri; Sat, 28 Dec 2019 11:33:55 +0000 Date: Sat, 28 Dec 2019 11:33:54 +0000 Message-ID: <861rsoit0d.wl-maz@kernel.org> From: Marc Zyngier To: Russell King Subject: Re: [PATCH] arm64: kvm: fix IDMAP overlap with HYP VA In-Reply-To: References: User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL/10.8 EasyPG/1.0.0 Emacs/26 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 62.31.163.78 X-SA-Exim-Rcpt-To: rmk+kernel@armlinux.org.uk, james.morse@arm.com, julien.thierry.kdev@gmail.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20191228_034304_448643_C59750C3 X-CRM114-Status: GOOD ( 29.42 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Suzuki K Poulose , Catalin Marinas , James Morse , Julien Thierry , Will Deacon , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Russell, Thanks for this. On Fri, 27 Dec 2019 11:47:35 +0000, Russell King wrote: > > Booting 5.4 on LX2160A reveals that KVM is non-functional: > > kvm: Limiting the IPA size due to kernel Virtual Address limit > kvm [1]: IPA Size Limit: 43bits > kvm [1]: IDMAP intersecting with HYP VA, unable to continue > kvm [1]: error initializing Hyp mode: -22 > > Debugging shows: > > kvm [1]: IDMAP page: 81a26000 > kvm [1]: HYP VA range: 0:22ffffffff > > as RAM is located at: > > 80000000-fbdfffff : System RAM > 2080000000-237fffffff : System RAM Ouch. This looks like a terrible choice for a memory map. > > Comparing this with the same kernel on Armada 8040 shows: > > kvm: Limiting the IPA size due to kernel Virtual Address limit > kvm [1]: IPA Size Limit: 43bits > kvm [1]: IDMAP page: 2a26000 > kvm [1]: HYP VA range: 4800000000:493fffffff > ... > kvm [1]: Hyp mode initialized successfully > > which indicates that hyp_va_msb is set, and is always set to the > opposite value of the idmap page to avoid the overlap. This does not > happen with the LX2160A. > > Further debugging shows vabits_actual = 39, kva_msb = 38 on LX2160A and > kva_msb = 33 on Armada 8040. Looking at the bit layout of the HYP VA, > there is still one bit available for hyp_va_msb. Set this bit > appropriately. This allows kvm to be functional on the LX2160A, but > without any HYP VA randomisation: > > kvm: Limiting the IPA size due to kernel Virtual Address limit > kvm [1]: IPA Size Limit: 43bits > kvm [1]: IDMAP page: 81a24000 > kvm [1]: HYP VA range: 4000000000:62ffffffff > ... > kvm [1]: Hyp mode initialized successfully Nice bit of debugging. I guess part of the confusion is due to the fact that the hyp_va_msb is part of the tag (as you found out), but that the documentation doesn't really make that clear at all (and only mentions the random part of the tag). > > Signed-off-by: Russell King > --- > arch/arm64/kvm/va_layout.c | 22 +++++++++++++++------- > 1 file changed, 15 insertions(+), 7 deletions(-) > > diff --git a/arch/arm64/kvm/va_layout.c b/arch/arm64/kvm/va_layout.c > index 2cf7d4b606c3..83f8b3f51cf4 100644 > --- a/arch/arm64/kvm/va_layout.c > +++ b/arch/arm64/kvm/va_layout.c > @@ -22,6 +22,17 @@ static u8 tag_lsb; > static u64 tag_val; > static u64 va_mask; > > +/* > + * We want to generate a hyp VA with the following format: > + * > + * 63 ... V | V-1 | V-2 .. tag_lsb | tag_lsb - 1 .. 0 > + * --------------------------------------------------------- > + * | 0000000 | hyp_va_msb | random tag | kern linear VA | \---------- tag -------------/ How about the above, to make it clearer that the tag must include the hyp_va_msb bit to avoid clashing with the IDMAP? > + * > + * which does not conflict with the idmap regions. This means that hyp_va_msb > + * must always be present. Luckily, when kva_msb == (vabits_actual - 1) we I'm not sure the "Luckily" part is appropriate here. Given the way kva_msb is computed, I can't see how it can see how this can fail. This stems from the fact that the vabits space for kernel mappings is split in two: vabits-1 for the linear map, and vabits-1 for the rest (kernel text and co). Given that we define kva_msb as the highest order bit that can change within the linear map, its value is at most vabits-1. > + * still have one bit for this, but no bits for the random tag. > + */ > static void compute_layout(void) > { > phys_addr_t idmap_addr = __pa_symbol(__hyp_idmap_text_start); > @@ -39,19 +50,16 @@ static void compute_layout(void) > /* > * No space in the address, let's compute the mask so > * that it covers (vabits_actual - 1) bits, and the region > - * bit. The tag stays set to zero. > + * bit. > */ > - va_mask = BIT(vabits_actual - 1) - 1; > - va_mask |= hyp_va_msb; > + tag_lsb = kva_msb; > + va_mask = BIT(vabits_actual - 1) - 1; > + tag_val = hyp_va_msb >> tag_lsb; > } else { > /* > * We do have some free bits to insert a random tag. > * Hyp VAs are now created from kernel linear map VAs > * using the following formula (with V == vabits_actual): > - * > - * 63 ... V | V-1 | V-2 .. tag_lsb | tag_lsb - 1 .. 0 > - * --------------------------------------------------------- > - * | 0000000 | hyp_va_msb | random tag | kern linear VA | > */ > tag_lsb = kva_msb; > va_mask = GENMASK_ULL(tag_lsb - 1, 0); In the light of this, it'd be great to rework this code to simplify it (getting rid of kva_msb, which really is tag_lsb) and make some of the computing common to both branches. Thanks, M. -- Jazz is not dead, it just smells funny. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel