From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6E418C433EF for ; Fri, 7 Jan 2022 18:20:41 +0000 (UTC) Received: from localhost ([::1]:44914 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1n5tpu-0000UN-Vs for qemu-devel@archiver.kernel.org; Fri, 07 Jan 2022 13:19:35 -0500 Received: from eggs.gnu.org ([209.51.188.92]:59050) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1n5toy-0008FD-1a for qemu-devel@nongnu.org; Fri, 07 Jan 2022 13:18:36 -0500 Received: from [2604:1380:4601:e00::1] (port=49774 helo=ams.source.kernel.org) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1n5tov-0000HE-W0 for qemu-devel@nongnu.org; Fri, 07 Jan 2022 13:18:35 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 1532BB82699; Fri, 7 Jan 2022 18:18:29 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C8318C36AE9; Fri, 7 Jan 2022 18:18:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1641579507; bh=cAh/1ukCeNr4ljylsYNoFGLIbakq3LEf6JKqjDyyNVk=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=N6GNW9TipkwfsVP0anEBua8YoApQratKn//cWadkH6KX41NT88vBgMoiNSJZelYE3 z7zxuaPWZJu8LTvbxPrTQKHw5JLYkJy4ujEHhxPFQGPw8H3V8H70tfp/mxnn+K+KKr TcRdGwbrridGXBVR4BlJ5l4FFHzwE0tvGxSLNsFhkojbNfVcTNgDoSWe0Nd4yiZS2+ mzBaneUvhUR+H23i2BzuUmeN6TaLwcbs785Pb9BlPGFLw8oCTS+bkH0gFgOiwqaa14 L7fe3vTjvUusnNQVj5o2KABlrPZ+IVbG5recQhB6aJxevugYnbRFw9yWaFdcus+6OT WWf8m4PMLA9xw== Received: from sofa.misterjones.org ([185.219.108.64] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1n5ton-00GcuU-LF; Fri, 07 Jan 2022 18:18:25 +0000 Date: Fri, 07 Jan 2022 18:18:25 +0000 Message-ID: <87y23rtnny.wl-maz@kernel.org> From: Marc Zyngier To: eric.auger@redhat.com Subject: Re: [PATCH v3 3/5] hw/arm/virt: Honor highmem setting when computing the memory map In-Reply-To: References: <20211227211642.994461-1-maz@kernel.org> <20211227211642.994461-4-maz@kernel.org> <871r1kzhbp.wl-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: eric.auger@redhat.com, qemu-devel@nongnu.org, drjones@redhat.com, peter.maydell@linaro.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, kernel-team@android.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-Host-Lookup-Failed: Reverse DNS lookup failed for 2604:1380:4601:e00::1 (failed) Received-SPF: pass client-ip=2604:1380:4601:e00::1; envelope-from=maz@kernel.org; helo=ams.source.kernel.org X-Spam_score_int: -16 X-Spam_score: -1.7 X-Spam_bar: - X-Spam_report: (-1.7 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.372, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RDNS_NONE=0.793, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Maydell , Andrew Jones , kvm@vger.kernel.org, qemu-devel@nongnu.org, kernel-team@android.com, kvmarm@lists.cs.columbia.edu Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Hi Eric, On Fri, 07 Jan 2022 17:15:19 +0000, Eric Auger wrote: > > Hi Marc, > > On 1/6/22 10:26 PM, Marc Zyngier wrote: > > On Wed, 05 Jan 2022 09:22:39 +0000, > > Eric Auger wrote: > >> Hi Marc, > >> > >> On 12/27/21 10:16 PM, Marc Zyngier wrote: > >>> Even when the VM is configured with highmem=off, the highest_gpa > >>> field includes devices that are above the 4GiB limit. > >>> Similarily, nothing seem to check that the memory is within > >>> the limit set by the highmem=off option. > >>> > >>> This leads to failures in virt_kvm_type() on systems that have > >>> a crippled IPA range, as the reported IPA space is larger than > >>> what it should be. > >>> > >>> Instead, honor the user-specified limit to only use the devices > >>> at the lowest end of the spectrum, and fail if we have memory > >>> crossing the 4GiB limit. > >>> > >>> Reviewed-by: Andrew Jones > >>> Signed-off-by: Marc Zyngier > >>> --- > >>> hw/arm/virt.c | 9 ++++++++- > >>> 1 file changed, 8 insertions(+), 1 deletion(-) > >>> > >>> diff --git a/hw/arm/virt.c b/hw/arm/virt.c > >>> index 8b600d82c1..84dd3b36fb 100644 > >>> --- a/hw/arm/virt.c > >>> +++ b/hw/arm/virt.c > >>> @@ -1678,6 +1678,11 @@ static void virt_set_memmap(VirtMachineState *vms) > >>> exit(EXIT_FAILURE); > >>> } > >>> > >>> + if (!vms->highmem && > >>> + vms->memmap[VIRT_MEM].base + ms->maxram_size > 4 * GiB) { > >>> + error_report("highmem=off, but memory crosses the 4GiB limit\n"); > >>> + exit(EXIT_FAILURE); > >> The memory is composed of initial memory and device memory. > >> device memory is put after the initial memory but has a 1GB alignment > >> On top of that you have 1G page alignment per device memory slot > >> > >> so potentially the highest mem address is larger than > >> vms->memmap[VIRT_MEM].base + ms->maxram_size. > >> I would rather do the check on device_memory_base + device_memory_size > > Yup, that's a good point. > > > > There is also a corner case in one of the later patches where I check > > this limit against the PA using the rounded-up device_memory_size. > > This could result in returning an error if the last memory slot would > > still fit in the PA space, but the rounded-up quantity wouldn't. I > > don't think it matters much, but I'll fix it anyway. > > > >>> + } > >>> /* > >>> * We compute the base of the high IO region depending on the > >>> * amount of initial and device memory. The device memory start/size > >>> @@ -1707,7 +1712,9 @@ static void virt_set_memmap(VirtMachineState *vms) > >>> vms->memmap[i].size = size; > >>> base += size; > >>> } > >>> - vms->highest_gpa = base - 1; > >>> + vms->highest_gpa = (vms->highmem ? > >>> + base : > >>> + vms->memmap[VIRT_MEM].base + ms->maxram_size) - 1; > >> As per the previous comment this looks wrong to me if !highmem. > > Agreed. > > > >> If !highmem, if RAM requirements are low we still could get benefit from > >> REDIST2 and HIGH ECAM which could fit within the 4GB limit. But maybe we > >> simply don't care? > > I don't see how. These devices live at a minimum of 256GB, which > > contradicts the very meaning of !highmem being a 4GB limit. > Yes I corrected the above statement afterwards, sorry for the noise. > > > >> If we don't, why don't we simply skip the extended_memmap overlay as > >> suggested in v2? I did not get your reply sorry. > > Because although this makes sense if you only care about a 32bit > > limit, we eventually want to check against an arbitrary PA limit and > > enable the individual devices that do fit in that space. > > In my understanding that is what virt_kvm_type() was supposed to do by > testing the result of kvm_arm_get_max_vm_ipa_size and requested_pa_size > (which accounted the high regions) and exiting if they were > incompatible. But I must miss something. This is a chicken and egg problem: you need the IPA size to compute the memory map, and you need the memory map to compute the IPA size. Fun, isn't it? At the moment, virt_set_memmap() doesn't know about the IPA space, generates a highest_gpa that may not work, and we end-up failing because the resulting VM type is out of bound. My solution to that is to feed the *maximum* IPA size to virt_set_memmap(), compute the memory map there, and then use highest_gpa to compute the actual IPA size that is used to create the VM. By knowing the IPA limit in virt_set_memmap(), I'm able to keep it in check and avoid generating an unusable memory map. I've tried to make that clearer in my v4. Hopefully I succeeded. Thanks, M. -- Without deviation from the norm, progress is not possible.