From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 74591C433E0 for ; Tue, 9 Mar 2021 14:48:23 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0640F64F62 for ; Tue, 9 Mar 2021 14:48:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0640F64F62 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Subject:Cc:To: From:Message-ID:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=LyvjL0LAYUyH41VRHpEpC9T49earA9kPIJPEblhWxQE=; b=DIXYYsvnQ1grk9WsK9AHoGxEb XUA35TtfMRIzxza20xzoGKNw9uASVja1TCNKLr8/yIHxmv4hTXuOuIOwWdqjFeqOtTDr+oqaOYqr7 an8LCzv++VnmLP2bVFxUjn4jmodK3qMTqU4rDijcNNyWybtuQwWZOWEzzEbuj0AN5CQ7/iWQK5Bc7 jx4uFchzlUbk3oYXaTydzGswuql2QgHstnoDOLR7JfpYSwVCNTXv1YHARZUa12dg+YELCP+cXkHj+ xmi55UHlQIr36L/gNhZXuOxFGZqeoJC0jSybOSnT4teqNsGIS9SqNjxm8i50kNCgRc24lH008uQtx qnxhSAeag==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lJddD-004sOm-6x; Tue, 09 Mar 2021 14:46:43 +0000 Received: from mail.kernel.org ([198.145.29.99]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lJdd6-004sMd-Vu for linux-arm-kernel@lists.infradead.org; Tue, 09 Mar 2021 14:46:41 +0000 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 6AE3B64F62; Tue, 9 Mar 2021 14:46:32 +0000 (UTC) Received: from 78.163-31-62.static.virginmediabusiness.co.uk ([62.31.163.78] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94) (envelope-from ) id 1lJdd0-000ZVc-5B; Tue, 09 Mar 2021 14:46:30 +0000 Date: Tue, 09 Mar 2021 14:46:29 +0000 Message-ID: <87k0qgxu4a.wl-maz@kernel.org> From: Marc Zyngier To: Andrew Jones Cc: kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, James Morse , Julien Thierry , Suzuki K Poulose , Alexandru Elisei , kernel-team@android.com Subject: Re: [PATCH] KVM: arm64: Cap default IPA size to the host's own size In-Reply-To: <20210309142910.awjhb52tgj5nxs72@kamzik.brq.redhat.com> References: <20210308174643.761100-1-maz@kernel.org> <20210309132021.7vuuf73joybhlhg3@kamzik.brq.redhat.com> <87mtvcxx0z.wl-maz@kernel.org> <20210309142910.awjhb52tgj5nxs72@kamzik.brq.redhat.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 62.31.163.78 X-SA-Exim-Rcpt-To: drjones@redhat.com, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, james.morse@arm.com, julien.thierry.kdev@gmail.com, suzuki.poulose@arm.com, alexandru.elisei@arm.com, kernel-team@android.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210309_144637_415268_9D0664A5 X-CRM114-Status: GOOD ( 46.16 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Tue, 09 Mar 2021 14:29:10 +0000, Andrew Jones wrote: > > On Tue, Mar 09, 2021 at 01:43:40PM +0000, Marc Zyngier wrote: > > Hi Andrew, > > > > On Tue, 09 Mar 2021 13:20:21 +0000, > > Andrew Jones wrote: > > > > > > Hi Marc, > > > > > > On Mon, Mar 08, 2021 at 05:46:43PM +0000, Marc Zyngier wrote: > > > > KVM/arm64 has forever used a 40bit default IPA space, partially > > > > due to its 32bit heritage (where the only choice is 40bit). > > > > > > > > However, there are implementations in the wild that have a *cough* > > > > much smaller *cough* IPA space, which leads to a misprogramming of > > > > VTCR_EL2, and a guest that is stuck on its first memory access > > > > if userspace dares to ask for the default IPA setting (which most > > > > VMMs do). > > > > > > > > Instead, cap the default IPA size to what the host can actually > > > > do, and spit out a one-off message on the console. The boot warning > > > > is turned into a more meaningfull message, and the new behaviour > > > > is also documented. > > > > > > > > Although this is a userspace ABI change, it doesn't really change > > > > much for userspace: > > > > > > > > - the guest couldn't run before this change, while it now has > > > > a chance to if the memory range fits the reduced IPA space > > > > > > > > - a memory slot that was accepted because it did fit the default > > > > IPA space but didn't fit the HW constraints is now properly > > > > rejected > > > > > > I'm not sure deferring the misconfiguration error until memslot > > > request time is better than just failing to create a VM. If > > > userspace doesn't use KVM_CAP_ARM_VM_IPA_SIZE to determine the > > > limit (which it hasn't been obliged to do) and it is able to > > > successfully create a VM, then it will assume up to 40-bit IPAs > > > are supported. Later, when it tries to add memslots and fails > > > it may be confused, especially if that later is much, much later > > > with memory hotplug. > > > > That's a fair point. However, no existing userspace will work on these > > systems. Is that what we want to do? I don't care much, but having > > non-usable defaults feel a bit... odd. I do spit out a warning, but I > > agree this isn't great either. > > I can send patches for QEMU, KVM selftests, and maybe even rust-vmm. > Can you point me to something about these systems I can reference > in my postings? Or I can just reference this mail thread. The system of choice to see this is an Apple M1 box. Not supported in mainline yet, but things are progressing pretty quickly. > > > > > > > The other thing that's left doing is to convince userspace to > > > > actually use the IPA space setting instead of relying on the > > > > antiquated default. > > > > > > Failing to create any VM which hasn't selected a valid IPA limit > > > should be pretty convincing :-) > > > > I'll make sure to redirect the reports your way! :D > > What's the current error message when this occurs? Is it good enough, or > should we improve it to help provide people hints? Please don't change > it to "Invalid IPA limit, please mail Andrew Jones" :-) Well, that's part of the problem. Currently, you don't get a message, and the guest faults on its first memory access forever (level 0 translation fault), as the VTCR_EL2.T0SZ value is bogus. I can change this patch to reject 40bit IPA when requested as a default with something saying "Userspace using unsupported default IPA limit, upgrade your VMM". Now, there is another nit[1] which I just found with my kvmtool setup that computes the optimal IPA space for a given VM. And that one is even more problematic... Thanks, M. [1] https://lore.kernel.org/r/87lfawxv40.wl-maz@kernel.org -- Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel