From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 252E6C433E7 for ; Mon, 19 Oct 2020 02:43:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D0867222C3 for ; Mon, 19 Oct 2020 02:43:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728953AbgJSCnR (ORCPT ); Sun, 18 Oct 2020 22:43:17 -0400 Received: from szxga06-in.huawei.com ([45.249.212.32]:57650 "EHLO huawei.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728791AbgJSCnR (ORCPT ); Sun, 18 Oct 2020 22:43:17 -0400 Received: from DGGEMS406-HUB.china.huawei.com (unknown [172.30.72.60]) by Forcepoint Email with ESMTP id 0E4BB91A978CEDFB0FE2; Mon, 19 Oct 2020 10:43:15 +0800 (CST) Received: from [10.174.176.61] (10.174.176.61) by DGGEMS406-HUB.china.huawei.com (10.3.19.206) with Microsoft SMTP Server id 14.3.487.0; Mon, 19 Oct 2020 10:43:07 +0800 Subject: Re: [PATCH v12 0/9] support reserving crashkernel above 4G on arm64 kdump To: Bhupesh Sharma , Catalin Marinas References: <20200907134745.25732-1-chenzhou10@huawei.com> <20201005170937.GA14576@gaia> <20201006180012.GB31946@C02TF0J2HF1T.local> CC: John Donnelly , Will Deacon , James Morse , Thomas Gleixner , "Ingo Molnar" , RuiRui Yang , Baoquan He , Jonathan Corbet , Prabhakar Kushwaha , Simon Horman , Rob Herring , Arnd Bergmann , , linux-arm-kernel , Linux Kernel Mailing List , kexec mailing list , Linux Doc Mailing List , , , , From: chenzhou Message-ID: Date: Mon, 19 Oct 2020 10:43:05 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.7.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit X-Originating-IP: [10.174.176.61] X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-doc@vger.kernel.org Hi Bhupesh, On 2020/10/7 15:07, Bhupesh Sharma wrote: > Hi Catalin, > > On Tue, Oct 6, 2020 at 11:30 PM Catalin Marinas wrote: >> On Mon, Oct 05, 2020 at 11:12:10PM +0530, Bhupesh Sharma wrote: >>> I think my earlier email with the test results on this series bounced >>> off the mailing list server (for some weird reason), but I still see >>> several issues with this patchset. I will add specific issues in the >>> review comments for each patch again, but overall, with a crashkernel >>> size of say 786M, I see the following issue: >>> >>> # cat /proc/cmdline >>> BOOT_IMAGE=(hd7,gpt2)/vmlinuz-5.9.0-rc7+ root=<..snip..> rd.lvm.lv=<..snip..> crashkernel=786M >>> >>> I see two regions of size 786M and 256M reserved in low and high >>> regions respectively, So we reserve a total of 1042M of memory, which >>> is an incorrect behaviour: >>> >>> # dmesg | grep -i crash >>> [ 0.000000] Reserving 256MB of low memory at 2816MB for crashkernel (System low RAM: 768MB) >>> [ 0.000000] Reserving 786MB of memory at 654158MB for crashkernel (System RAM: 130816MB) >>> [ 0.000000] Kernel command line: BOOT_IMAGE=(hd2,gpt2)/vmlinuz-5.9.0-rc7+ root=/dev/mapper/rhel_ampere--hr330a--03-root ro rd.lvm.lv=rhel_ampere-hr330a-03/root rd.lvm.lv=rhel_ampere-hr330a-03/swap crashkernel=786M cma=1024M >>> >>> # cat /proc/iomem | grep -i crash >>> b0000000-bfffffff : Crash kernel (low) >>> bfcbe00000-bffcffffff : Crash kernel >> As Chen said, that's the intended behaviour and how x86 works. The >> requested 768M goes in the high range if there's not enough low memory >> and an additional buffer for swiotlb is allocated, hence the low 256M. > I understand, but why 256M (as low) for arm64? x86_64 setups usually > have more system memory available as compared to several commercially > available arm64 setups. So is the intent, just to keep the behavior > similar between arm64 and x86_64? > > Should we have a CONFIG option / bootarg to help one select the max > 'low_size'? Currently the ' low_size' value is calculated as: > > /* > * two parts from kernel/dma/swiotlb.c: > * -swiotlb size: user-specified with swiotlb= or default. > * > * -swiotlb overflow buffer: now hardcoded to 32k. We round it > * to 8M for other buffers that may need to stay low too. Also > * make sure we allocate enough extra low memory so that we > * don't run out of DMA buffers for 32-bit devices. > */ > low_size = max(swiotlb_size_or_default() + (8UL << 20), 256UL << 20); > > Since many arm64 boards ship with swiotlb=0 (turned off) via kernel > bootargs, the low_size, still ends up being 256M in such cases, > whereas this 256M can be used for some other purposes - so should we > be limiting this to 64M and failing the crash kernel allocation > request (gracefully) otherwise? > >> We could (as an additional patch), subtract the 256M from the high >> allocation so that you'd get a low 256M and a high 512M, not sure it's >> worth it. Note that with a "crashkernel=768M,high" option, you still get >> the additional low 256M, otherwise the crashkernel won't be able to >> boot as there's no memory in ZONE_DMA. In the explicit ",high" request >> case, I'm not sure subtracted the 256M is more intuitive. >> In 5.11, we also hope to fix the ZONE_DMA layout for non-RPi4 platforms >> to cover the entire 32-bit address space (i.e. identical to the current >> ZONE_DMA32). >> >>> IMO, we should test this feature more before including this in 5.11 >> Definitely. That's one of the reasons we haven't queued it yet. So any >> help with testing here is appreciated. > Sure, I am running more checks on this series. I will be soon back > with more updates. Sorry to bother you. I am looking forward to your review comments. Thanks, Chen Zhou > > Regards, > Bhupesh > > . >