From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 78935ECAAD3 for ; Thu, 1 Sep 2022 07:25:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D2CD48D0001; Thu, 1 Sep 2022 03:25:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CDC656B0073; Thu, 1 Sep 2022 03:25:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BA43A8D0001; Thu, 1 Sep 2022 03:25:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id AA5826B0072 for ; Thu, 1 Sep 2022 03:25:25 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 75B0540158 for ; Thu, 1 Sep 2022 07:25:25 +0000 (UTC) X-FDA: 79862681010.15.8F99EB8 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf15.hostedemail.com (Postfix) with ESMTP id A3578A0041 for ; Thu, 1 Sep 2022 07:25:23 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sin.source.kernel.org (Postfix) with ESMTPS id 61C9ECE227B; Thu, 1 Sep 2022 07:25:19 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B3F13C433C1; Thu, 1 Sep 2022 07:25:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1662017117; bh=wFBxgLE3ZKENwiyGQvZkmf0lQ+nkRd9E9wQrdKxQ/pw=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=Nx+Vz2yadBlXqmB6mN0pWinuEwxeVrsKNBWvD2k0uXISs3cAMKJhrrCZmZEs6gu31 hYnTLg8w3rxCg8EtPBH/PCVvX+tGAU8LPYyLD3GyyZp6vGjULev/k40TaOSbXWMr3Z jLUgV6aDdN4jifZ4BjpI0kHDwTQOUHe1GN1MznmfXdNVwXmk0U94fH6jjnj0MH5mPE uzB7AClQ26HdjHL8GdHq54asN0puEwicseuMkKbL0G7uW4KFJjk+oxddovX+FwBr4Z 9MVTdOTJ/ATlVrqMU9Mh4fUemQ4kEtEFColcwKWqiVNxXMWxq9YhFjjdebynIygckr vG0iaJFB2ZrLg== Date: Thu, 1 Sep 2022 10:24:59 +0300 From: Mike Rapoport To: Baoquan He Cc: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, catalin.marinas@arm.com, ardb@kernel.org, guanghuifeng@linux.alibaba.com, mark.rutland@arm.com, will@kernel.org, linux-mm@kvack.org, thunder.leizhen@huawei.com, wangkefeng.wang@huawei.com, kexec@lists.infradead.org Subject: Re: [PATCH 1/2] arm64, kdump: enforce to take 4G as the crashkernel low memory end Message-ID: References: <20220828005545.94389-1-bhe@redhat.com> <20220828005545.94389-2-bhe@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1662017124; a=rsa-sha256; cv=none; b=PSPtsryx/kSRpnLJ93T6lixjuWWqo7MhSj2nPn4oK/zu4agQsOjD0ujE32oAn/xlEELwcK IKgQiqew91J8vI6PWDYyNQ6+sIjGlQYdXsVThERvzLFL7uDU2kAjduP7I1klBDNSrD7Y0T 7wsBqzay/dNEbENhd4J7WI6gZdic5Cw= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Nx+Vz2ya; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf15.hostedemail.com: domain of rppt@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=rppt@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1662017124; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=rI+zv0pppg0HYaoxYFfemsXg2XfXn32Q7shzvtWhk7U=; b=SUlQOAKyMW8YeqKZu9lxnDisFq4j5By/2b2ujbDapWIGJVQV4T31bIhJ1KcBZtOs1+GNaz oe2SNvv6co1tgnH4fxUw1NJBmj5iZpFHv9OGDdvthNVEeR/PT0P9onnrzhEJ6jxtp4q0GB QaGW84thfdVO5Axc0fCdmhXL+0cH6Qw= X-Rspamd-Queue-Id: A3578A0041 Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Nx+Vz2ya; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf15.hostedemail.com: domain of rppt@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=rppt@kernel.org X-Rspam-User: X-Rspamd-Server: rspam06 X-Stat-Signature: 43du83ouoyxrpfur73uxfc95gcd44di9 X-HE-Tag: 1662017123-143017 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Aug 31, 2022 at 10:29:39PM +0800, Baoquan He wrote: > On 08/31/22 at 10:37am, Mike Rapoport wrote: > > On Sun, Aug 28, 2022 at 08:55:44AM +0800, Baoquan He wrote: > > > > > > Solution: > > > ========= > > > To fix the problem, we should always take 4G as the crashkernel low > > > memory end in case CONFIG_ZONE_DMA or CONFIG_ZONE_DMA32 is enabled. > > > With this, we don't need to defer the crashkernel reservation till > > > bootmem_init() is called to set the arm64_dma_phys_limit. As long as > > > memblock init is done, we can conclude what is the upper limit of low > > > memory zone. > > > > > > 1) both CONFIG_ZONE_DMA or CONFIG_ZONE_DMA32 are disabled or memblock_start_of_DRAM() > 4G > > > limit = PHYS_ADDR_MAX+1 (Corner cases) > > > > Why these are corner cases? > > The case when CONFIG_ZONE_DMA or CONFIG_ZONE_DMA32 are disabled is the > > simplest one because it does not require the whole dancing around > > arm64_dma_phys_limit initialization. > > > > And AFAIK, memblock_start_of_DRAM() > 4G is not uncommon on arm64, but it > > does not matter for device DMA addressing. > > Thanks for reviewing. > > I could be wrong and have misunderstanding about corner case. > > With my understanding, both ZONE_DMA and ZONE_DMA32 are enabled by > default in kernel. And on distros, I believe they are on too. The both > ZONE_DMA and ZONE_DMA32 disabled case should only exist on one specific > product, and the memblock_start_of_DRAM() > 4G case too. At least, I > haven't seen one in our LAB. What I thought the non generic as corner > case could be wrong. I will change that phrasing. > > mm/Kconfig: > config ZONE_DMA > bool "Support DMA zone" if ARCH_HAS_ZONE_DMA_SET > default y if ARM64 || X86 > > config ZONE_DMA32 > bool "Support DMA32 zone" if ARCH_HAS_ZONE_DMA_SET > depends on !X86_32 > default y if ARM64 My point was that the cases with ZONE_DMA/DMA32 disabled or with RAM above 4G do not require detection of arm64_dma_phys_limit before reserving the crash kernel, can use predefined constants and are simple to handle. > > The actual corner cases are systems with ZONE_DMA/DMA32 and with <32 bits > > limit for device DMA addressing (e.g RPi 4). I think the changelog should > > Right, RPi4's 30bit DMA addressing device is corner case. > > > mention that to use kdump on these devices user must specify > > crashkernel=X@Y > > Makes sense. I will add words in log, and add sentences to > mention that in code comment or some place of document. > Thanks for advice. > > > > > > 2) CONFIG_ZONE_DMA or CONFIG_ZONE_DMA32 are enabled: > > > limit = 4G (generic case) > > > ... > > > +static phys_addr_t __init crash_addr_low_max(void) > > > +{ > > > + phys_addr_t low_mem_mask = U32_MAX; > > > + phys_addr_t phys_start = memblock_start_of_DRAM(); > > > + > > > + if ((!IS_ENABLED(CONFIG_ZONE_DMA) && !IS_ENABLED(CONFIG_ZONE_DMA32)) || > > > + (phys_start > U32_MAX)) > > > + low_mem_mask = PHYS_ADDR_MAX; > > > + > > > + return min(low_mem_mask, memblock_end_of_DRAM() - 1) + 1; > > > > Since RAM frequently starts on non-zero address the limit for systems with > > ZONE_DMA/DMA32 should be memblock_start_of_DRAM() + 4G. There is no need to > > It may not be right for memblock_start_of_DRAM(). On most of arm64 > servers I ever tested, their memblock usually starts from a higher > address, but not zero which is like x86. E.g below memory ranges printed > on an ampere-mtsnow-altra system, the starting addr is 0x83000000. With > my understanding, DMA addressing bits correspond to the cpu logical > address range devices can address. So memblock_start_of_DRAM() + 4G > seems not right for normal system, and not right for system which > starting physical address is above 4G. I refer to max_zone_phys() of > arch/arm64/mm/init.c when implementing crash_addr_low_max(). Please > correct me if I am wrong. My understanding was that no matter where DRAM starts, the first 4G would be accessible by 32-bit devices, but I maybe wrong as well :) I haven't notice you used max_zone_phys() as a reference. Wouldn't it be simpler to just call it from crash_addr_low_max(): static phys_addr_t __init crash_addr_low_max(void) { return max_zone_phys(32); } -- Sincerely yours, Mike.