From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1B76EC433EF for ; Thu, 5 May 2022 03:04:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=UFIGuehNDuSevVP5QH5GxWjc2B5K3XrrLmlmf6kn01A=; b=Vdu4gCzM4AE9bN KddxgJwknvS5cdCaXnp+o20bshf+jvZ03TyTW0phDasmQ9SKgQa3QV/CtPmmSmDsqb7z5tny9Gbzq 5Xjf3Edtkjb7leC1dncSSferqVyy6Nqg+F5/o6JzGNzsIICyrHilAqW5YVxre4KloOQGLOEYZR44+ uFmEvLbRKk3jeyvH4sWSOS5TQrq3kSzRWADBSPqF/y6p4x0JpoGGy/wwIrltAbNKGH5gI+evgswod GDy8bIHPIe1y4aCqGJQXTAy9upIAxKPaxxLS68NczGItxfyLDqudCmV3WY//3bQRxVRBaqPGRCGT0 gcYAj4I0U+A8fPwNXu7Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nmRll-00DiQM-L8; Thu, 05 May 2022 03:03:09 +0000 Received: from us-smtp-delivery-74.mimecast.com ([170.10.133.74]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nmRlg-00DiP3-2e for linux-arm-kernel@lists.infradead.org; Thu, 05 May 2022 03:03:06 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1651719781; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=kXbvduo1+pXP2xsQyJhB2ESjMhg7t8jVXPIOUihsXiA=; b=EWCP0DaCAUeeR1o8+gN0ZTcSK/gwMaXO7YfxGz2k4ZxXOuIdVIemCfNZXTGO081tLY1eQh AEyZCts4vYhKIn6nG6PRRIsJ8sH7zoqGeJ1nPX695qJoVKMzwjydRWU5lXF23S3jVw7Bdv mVKio7ZoPxnRvULGF/Z7E/aI7QHZBX0= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-352-2UwQDUfUOkCZSKV-5UDNDQ-1; Wed, 04 May 2022 23:00:25 -0400 X-MC-Unique: 2UwQDUfUOkCZSKV-5UDNDQ-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 6FF7038041C1; Thu, 5 May 2022 03:00:24 +0000 (UTC) Received: from localhost (ovpn-12-197.pek2.redhat.com [10.72.12.197]) by smtp.corp.redhat.com (Postfix) with ESMTPS id C29EF40D2822; Thu, 5 May 2022 03:00:22 +0000 (UTC) Date: Thu, 5 May 2022 11:00:19 +0800 From: Baoquan He To: Catalin Marinas Cc: "Leizhen (ThunderTown)" , Thomas Gleixner , Ingo Molnar , Borislav Petkov , x86@kernel.org, "H . Peter Anvin" , linux-kernel@vger.kernel.org, Dave Young , Vivek Goyal , Eric Biederman , kexec@lists.infradead.org, Will Deacon , linux-arm-kernel@lists.infradead.org, Rob Herring , Frank Rowand , devicetree@vger.kernel.org, Jonathan Corbet , linux-doc@vger.kernel.org, Randy Dunlap , Feng Zhou , Kefeng Wang , Chen Zhou , John Donnelly , Dave Kleikamp Subject: Re: [PATCH v22 5/9] arm64: kdump: Reimplement crashkernel=X Message-ID: References: <3fc41a94-4247-40f3-14e7-f11e3001ec33@huawei.com> <23e2dcf4-4e9a-5298-d5d8-8761b0bbbe21@huawei.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220504_200304_287910_7934F41E X-CRM114-Status: GOOD ( 51.47 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 05/03/22 at 11:00pm, Catalin Marinas wrote: > On Fri, Apr 29, 2022 at 04:25:37PM +0800, Leizhen (ThunderTown) wrote: > > On 2022/4/29 16:02, Leizhen (ThunderTown) wrote: > > > On 2022/4/29 11:24, Baoquan He wrote: > > >> On 04/28/22 at 05:33pm, Leizhen (ThunderTown) wrote: > > >>> On 2022/4/28 11:52, Baoquan He wrote: > > >>>> On 04/28/22 at 11:40am, Baoquan He wrote: > > >>>>> On 04/27/22 at 05:04pm, Catalin Marinas wrote: > > >>>>>> There will be some difference as the 4G limit doesn't always hold for > > >>>>>> arm64 (though it's true in most cases). Anyway, we can probably simplify > > >>>>>> things a bit while following the documented behaviour: > > >>>>>> > > >>>>>> crashkernel=Y - current behaviour within ZONE_DMA > > >>>>>> crashkernel=Y,high - allocate from above ZONE_DMA > > >>>>>> crashkernel=Y,low - allocate within ZONE_DMA > [...] > > >>>>> Sorry to interrupt. Seems the ,high ,low and fallback are main concerns > > >>>>> about this version. And I have the same concerns about them which comes > > >>>>> from below points: > > >>>>> 1) we may need to take best effort to keep ,high, ,low behaviour > > >>>>> consistent on all ARCHes. Otherwise user/admin may be confused when they > > >>>>> deploy/configure kdump on different machines of different ARCHes in the > > >>>>> same LAB. I think we should try to avoid the confusion. > > I guess by all arches you mean just x86 here. Since the code is not > generic, all arches do their own stuff. Right. Since currently only x86 has crashkernel,high|low support. From the distros and customer's point of view, we would like to see the same feature has the same or similar behaviour. This will ease operation and maintaining. E.g on the cloud platform, the base of it could be any ARCH, x86, arm64. The inconsistent behaviour could cause confusion. Certainly, the underlying implementation may be different. Surely, if arm64 has its own manner because of reasons, we can add document to note that. > > > > OK, I plan to remove optimization, fallback and default low size, to follow the > > > suggestion of Catalin first. But there's one minor point of contention. > > > > > > 1) Both "crashkernel=X,high" and "crashkernel=X,low" must be present. > > > 2) Both "crashkernel=X,high" and "crashkernel=X,low" are present. > > > or > > > Allow "crashkernel=X,high" to be present alone. Unlike x86, the default low size is zero. > > > > > > I prefer 2), how about you? > > (2) works for me as well. We keep these simple as "expert" options and > allow crashkernel= to fall back to 'high' if not sufficient memory in > ZONE_DMA. That would be a slight change from the current behaviour but, > as Zhen Lei said, with the old tools it's just moving the error around, > the crashkernel wouldn't be available in either case. > > > >>>>> 2) Fallback behaviour is important to our distros. The reason is we will > > >>>>> provide default value with crashkernel=xxxM along kernel of distros. In > > >>>>> this case, we hope the reservation will succeed by all means. The ,high > > >>>>> and ,low is an option if customer likes to take with expertise. > > OK, that's good feedback. > > So, to recap, IIUC you are fine with: > > crashkernel=Y - allocate within ZONE_DMA with fallback > above with a default in ZONE_DMA (like > x86, 256M or swiotlb size) Ack to this one. > crashkernel=Y,high - allocate from above ZONE_DMA Not exactly. If there's only ZONE_DMA, crashkernel,high will be reserved in ZONE_DMA, and crashkernel,low will be ignored. Other than this, ack. > crashkernel=Y,low - allocate within ZONE_DMA Ack to this one. > > 'crashkernel' overrides the high and low while the latter two can be > passed independently. crashkernel=,high can be passed independently, then a crashkernel=,low is needed implicitly. If people don't want crashkernel=,low explicitly, crashkernel=0,low need be specified. An independent crashkernel=,low makes no sense. Crashkernel=,low should be paird with crashkernel=,high. My personal opinion according to the existed senmantics on x86. Otherwise, the guidance of crashkernel= |,high|,low reservation will be complicated to write. > > > >>>>> After going through arm64 memory init code, I got below summary about > > >>>>> arm64_dma_phys_limit which is the first zone's upper limit. I think we > > >>>>> can make use of it to facilitate to simplify code. > > >>>>> ================================================================================ > > >>>>> DMA DMA32 NORMAL > > >>>>> 1)Raspberry Pi4 0~1G 3G~4G (above 4G) > > >>>>> 2)Normal machine 0~4G 0 (above 4G) > > >>>>> 3)Special machine (above 4G)~MAX > > >>>>> 4)No DMA|DMA32 (above 4G)~MAX > > >>> > > >>> arm64_memblock_init() > > >>> reserve_crashkernel() --------------- 0a30c53573b0 ("arm64: mm: Move reserve_crashkernel() into mem_init()") > > >> We don't need different code for this place of reservation as you are > > >> doing in this patchset, since arm64_dma_phys_limit is initialized as > > >> below. In fact, in arm64_memblock_init(), we have made memblock ready, > > >> we can initialize arm64_dma_phys_limit as memblock_end_of_DRAM(). And if > > >> memblock_start_of_DRAM() is bigger than 4G, we possibly can call > > >> reserve_crashkernel() here too. > > > > > > Yes. Maybe all the devices in this environment are 64-bit. One way I > > > know of allowing 32-bit devices to access high memory without SMMU > > > is: Set a fixed value for the upper 32 bits. In this case, the DMA > > > zone should be [phys_start, phys_start + 4G). > > We decided that this case doesn't really exists for arm64 platforms (no > need for special ZONE_DMA). > > > I just read the message of commit 791ab8b2e3 ("arm64: Ignore any DMA > > offsets in the max_zone_phys() calculation") > > > > Currently, the kernel assumes that if RAM starts above 32-bit (or > > zone_bits), there is still a ZONE_DMA/DMA32 at the bottom of the RAM and > > such constrained devices have a hardwired DMA offset. In practice, we > > haven't noticed any such hardware so let's assume that we can expand > > ZONE_DMA32 to the available memory if no RAM below 4GB. Similarly, > > ZONE_DMA is expanded to the 4GB limit if no RAM addressable by > > zone_bits. > > I think the above log is slightly confusing. If the DRAM starts above > 4G, ZONE_DMA goes to the end of DRAM. If the DRAM starts below 4G but > above the zone_bits for ZONE_DMA as specified in DT/ACPI, it pushes > ZONE_DMA to 4G. I don't remember why we did this last part, maybe in > case we get incorrect firmware tables, otherwise we could have extended > ZONE_DMA to end of DRAM. > > Zhen Lei, if we agreed on the crashkernel behaviour, could you please > post a series that does the above parsing allocation? Ignore the > optimisations, we can look at them afterwards. > > Thanks. > > -- > Catalin > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel