From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 761CCC4167B for ; Wed, 29 Nov 2023 07:58:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=j/TXnoL+TysE3XhJBiNtCgbC+Edp4sdler76Mrc44lQ=; b=DHXPdb4PeN62a2 gd0h4NOs0jq+SRkDiAk8KSbIqcTvtvKIk2QsVDu+mvCAzF/gyVfQXmfKqa1BwDScSqeTxQq/5XHee D0McJZOgRNMsVekFnnK3rvBZbYx+cg9GcB8vkE3cu+5u29kxqhZbIj74KEwTEGWsAGMTydO0CSYcr 4TAkNetIRtt+7LTqX66chCFQH5jYgRDNfihQjLrb2CFP/Q0LnBIQu6mKZf1Xlf92uv9nORpuaFcPL H3yoBkXP1Sq6hcfj9gfZ67b8MAaa5DAPXRmIuO2pBLeHTF2UwyIFOEgixazatT4GTwhDcRdRZX9Eo e1vwksIOB14SmZ4jmKkA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1r8FSW-007PQn-0q; Wed, 29 Nov 2023 07:58:12 +0000 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1r8FST-007PQ1-0f for kexec@lists.infradead.org; Wed, 29 Nov 2023 07:58:10 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1701244687; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=8g8v5NESvRdEm67kdhAr+zNdrT8GqKRlUvJtdOuI41I=; b=c/Aw02zrtWtnfPahmjwKwtPwNOR7B0hYgMTUS1dyYXW6xZyB9ZNJNEMw1owXwH51y/sDas t/ZV9QIo8IZ99BycemWMuwzBOktzBHWr6BOiRh6qWDVRsI5I4dIBHH2SuhXLmCujL2S4uB ShUXw9Kq24jTmralkQCVqL4yEwbT/zY= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-662-m1DTePDEO8qKOmsP5YTK1Q-1; Wed, 29 Nov 2023 02:58:03 -0500 X-MC-Unique: m1DTePDEO8qKOmsP5YTK1Q-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 56053811E7D; Wed, 29 Nov 2023 07:58:03 +0000 (UTC) Received: from localhost (unknown [10.72.112.30]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 84923C1596F; Wed, 29 Nov 2023 07:58:02 +0000 (UTC) Date: Wed, 29 Nov 2023 15:57:59 +0800 From: Baoquan He To: Michal Hocko Cc: Jiri Bohac , Pingfan Liu , Tao Liu , Vivek Goyal , Dave Young , kexec@lists.infradead.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH 0/4] kdump: crashkernel reservation from CMA Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.8 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231128_235809_336128_A2D01BC1 X-CRM114-Status: GOOD ( 34.32 ) X-BeenThere: kexec@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "kexec" Errors-To: kexec-bounces+kexec=archiver.kernel.org@lists.infradead.org On 11/28/23 at 10:08am, Michal Hocko wrote: > On Tue 28-11-23 10:11:31, Baoquan He wrote: > > On 11/28/23 at 09:12am, Tao Liu wrote: > [...] > > Thanks for the effort to bring this up, Jiri. > > > > I am wondering how you will use this crashkernel=,cma parameter. I mean > > the scenario of crashkernel=,cma. Asking this because I don't know how > > SUSE deploy kdump in SUSE distros. In SUSE distros, kdump kernel's > > driver will be filter out? If latter case, It's possibly having the > > on-flight DMA issue, e.g NIC has DMA buffer in the CMA area, but not > > reset during kdump bootup because the NIC driver is not loaded in to > > initialize. Not sure if this is 100%, possible in theory? > > NIC drivers do not allocation from movable zones (that includes CMA > zone). In fact kernel doesn't use GFP_MOVABLE for non-user requests. > RDMA drivers might and do transfer from user backed memory but for that > purpose they should be pinning memory (have a look at > __gup_longterm_locked and its callers) and that will migrate away from > the any zone. OK, in that case, we don't need to worry about the risk of DMA. > > [...] > > The crashkernel=,cma requires no userspace data dumping, from our > > support engineers' feedback, customer never express they don't need to > > dump user space data. Assume a server with huge databse deployed, and > > the database often collapsed recently and database provider claimed that > > it's not database's fault, OS need prove their innocence. What will you > > do? > > Don't use CMA backed crash memory then? This is an optional feature. Guess so. As I said earlier, this is more like a nice-to-have feature, can suggest user to try by themselves. Since Jiri didn't give how he will use it. > > > So this looks like a nice to have to me. At least in fedora/rhel's > > usage, we may only back port this patch, and add one sentence in our > > user guide saying "there's a crashkernel=,cma added, can be used with > > crashkernel= to save memory. Please feel free to try if you like". > > Unless SUSE or other distros decides to use it as default config or > > something like that. Please correct me if I missed anything or took > > anything wrong. > > Jiri will know better than me but for us a proper crash memory > configuration has become a real nut. You do not want to reserve too much > because it is effectively cutting of the usable memory and we regularly > hit into "not enough memory" if we tried to be savvy. The more tight you > try to configure the easier to fail that is. Even worse any in kernel > memory consumer can increase its memory demand and get the overall > consumption off the cliff. So this is not an easy to maintain solution. > CMA backed crash memory can be much more generous while still usable. Hmm, Redhat could go in a different way. We have been trying to: 1) customize initrd for kdump kernel specifically, e.g exclude unneeded devices's driver to save memory; 2) monitor device and kenrel memory usage if they begin to consume much more memory than before. We have CI testing cases to watch this. We ever found one NIC even eat up GB level memory, then this need be investigated and fixed. With these effort, our default crashkernel values satisfy most of cases, surely not call cases. Only rare cases need be handled manually, increasing crashkernel. The crashkernel=,high was added in this case, a small low memory under 4G for DMA with crashkernel=,low, a big chunk of high memory above 4G with crashkernel=,high. I can't see where needs are not met. Wondering how you will use this crashkernel=,cma syntax. On normal machines and virt guests, not much meomry is needed, usually 256M or a little more is enough. On those high end systems with hundreds of Giga bytes, even Tera bytes of memory, I don't think the saved memory with crashkernel=,cma make much sense. Taking out 1G memory above 4G as crashkernel won't impact much. So with my understanding, crashkernel=,cma adds an option user can take besides the existing crashkernel=,high. As I have said earlier, in Redhat, we may rebase it to fedora/RHEL and add one sentence into our user guide saying "one another crashkernel=,cma can be use to save memory, please feel free to try if you like." Then that's it. Guess SUSE will check user's configuration, e.g the dump level of makedumpfile, if no user space data needed, crashkernel=,cma is taken, otherwise the normal crashkernel=xM will be chosen? Thanks Baoquan _______________________________________________ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec