From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 34D0CC433DF for ; Mon, 27 Jul 2020 14:40:59 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 048F92083B for ; Mon, 27 Jul 2020 14:40:59 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="SVM6vmLo" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 048F92083B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.ibm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:References:In-Reply-To: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=qaYWhD0vbj/v6P9xo6E8Ty29vkjCfOb4xttD4Rz/MEQ=; b=SVM6vmLoosT8Vhq4olpeo+6nC HHO8EXfXKnT23/efhVdPRjtUm+9UjHoomxoT+WaC70c5hcYxrJQ2c8wPNXnOwad3LuwafgMvaKXgq HvBuE4GsFbol9FpNjIcTnBotb1YB1pfZ/L9d3qrRKkp2RyySGkjNvMyjKL6rUGXo+Ebr7Pet95qja G/SDqCEt6MhMVf8DMnIdTxzi1El2+sLelhEKlh1opF2L9xB7Tc5uTaFAIyCIqNMHngu7mh7Yii2gJ q6AHNOFcbf/5XdmB6p+dY0Vnr1M91hLU4BOWYXuUnD9317dhfs091sxV98h9PcApcOi28HmV8rXBF xGMBGTr6w==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1k04HA-00011n-TI; Mon, 27 Jul 2020 14:38:48 +0000 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5] helo=mx0a-001b2d01.pphosted.com) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1k04H7-00010e-Uh for linux-arm-kernel@lists.infradead.org; Mon, 27 Jul 2020 14:38:47 +0000 Received: from pps.filterd (m0098416.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 06REWGiC063319; Mon, 27 Jul 2020 10:38:01 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 32hrcyfspx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 27 Jul 2020 10:38:01 -0400 Received: from m0098416.ppops.net (m0098416.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.36/8.16.0.36) with SMTP id 06REYfZu074767; Mon, 27 Jul 2020 10:38:00 -0400 Received: from ppma01dal.us.ibm.com (83.d6.3fa9.ip4.static.sl-reverse.com [169.63.214.131]) by mx0b-001b2d01.pphosted.com with ESMTP id 32hrcyfspf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 27 Jul 2020 10:38:00 -0400 Received: from pps.filterd (ppma01dal.us.ibm.com [127.0.0.1]) by ppma01dal.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 06REZGnr017269; Mon, 27 Jul 2020 14:37:59 GMT Received: from b01cxnp23033.gho.pok.ibm.com (b01cxnp23033.gho.pok.ibm.com [9.57.198.28]) by ppma01dal.us.ibm.com with ESMTP id 32gcy2x7wa-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 27 Jul 2020 14:37:59 +0000 Received: from b01ledav003.gho.pok.ibm.com (b01ledav003.gho.pok.ibm.com [9.57.199.108]) by b01cxnp23033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 06REbwP559441424 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 27 Jul 2020 14:37:59 GMT Received: from b01ledav003.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E0DFDB2064; Mon, 27 Jul 2020 14:37:58 +0000 (GMT) Received: from b01ledav003.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A07F8B205F; Mon, 27 Jul 2020 14:37:53 +0000 (GMT) Received: from skywalker.linux.ibm.com (unknown [9.79.210.187]) by b01ledav003.gho.pok.ibm.com (Postfix) with ESMTP; Mon, 27 Jul 2020 14:37:53 +0000 (GMT) X-Mailer: emacs 27.0.91 (via feedmail 11-beta-1 I) From: "Aneesh Kumar K.V" To: Mike Kravetz , Anshuman Khandual , Will Deacon , Roman Gushchin Subject: Re: [PATCH v3] mm/hugetlb: split hugetlb_cma in nodes with memory In-Reply-To: References: <20200710120950.37716-1-song.bao.hua@hisilicon.com> <359ea1d0-b1fd-d09f-d28a-a44655834277@oracle.com> <20200715081822.GA5683@willie-the-truck> <5724f1f8-63a6-ee0f-018c-06fb259b6290@oracle.com> <20200716081243.GA6561@willie-the-truck> <81103d30-f4fd-8807-03f9-d131da5097bd@arm.com> <1efdfe52-abdb-3931-742c-70e4a170e403@oracle.com> <11b03fcd-c210-032c-16d2-79ada41e0349@arm.com> Date: Mon, 27 Jul 2020 20:07:51 +0530 Message-ID: <874kptm3b4.fsf@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235, 18.0.687 definitions=2020-07-27_08:2020-07-27, 2020-07-27 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 malwarescore=0 suspectscore=0 adultscore=0 mlxlogscore=999 priorityscore=1501 spamscore=0 clxscore=1011 mlxscore=0 phishscore=0 bulkscore=0 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2007270103 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200727_103846_071441_90C1D568 X-CRM114-Status: GOOD ( 28.08 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Barry Song , "H.Peter Anvin" , Catalin Marinas , x86@kernel.org, linux-kernel@vger.kernel.org, linuxarm@huawei.com, linux-mm@kvack.org, Ingo Molnar , Borislav Petkov , Jonathan Cameron , Thomas Gleixner , Mike Rapoport , akpm@linux-foundation.org, linux-arm-kernel@lists.infradead.org Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Mike Kravetz writes: > On 7/19/20 11:22 PM, Anshuman Khandual wrote: >> >> >> On 07/17/2020 10:32 PM, Mike Kravetz wrote: >>> On 7/16/20 10:02 PM, Anshuman Khandual wrote: >>>> >>>> >>>> On 07/16/2020 11:55 PM, Mike Kravetz wrote: >>>>> >From 17c8f37afbf42fe7412e6eebb3619c6e0b7e1c3c Mon Sep 17 00:00:00 2001 >>>>> From: Mike Kravetz >>>>> Date: Tue, 14 Jul 2020 15:54:46 -0700 >>>>> Subject: [PATCH] hugetlb: move cma reservation to code setting up gigantic >>>>> hstate >>>>> >>>>> Instead of calling hugetlb_cma_reserve() directly from arch specific >>>>> code, call from hugetlb_add_hstate when adding a gigantic hstate. >>>>> hugetlb_add_hstate is either called from arch specific huge page setup, >>>>> or as the result of hugetlb command line processing. In either case, >>>>> this is late enough in the init process that all numa memory information >>>>> should be initialized. And, it is early enough to still use early >>>>> memory allocator. >>>> >>>> This assumes that hugetlb_add_hstate() is called from the arch code at >>>> the right point in time for the generic HugeTLB to do the required CMA >>>> reservation which is not ideal. I guess it must have been a reason why >>>> CMA reservation should always called by the platform code which knows >>>> the boot sequence timing better. >>> >>> Actually, the code does not make the assumption that hugetlb_add_hstate >>> is called from arch specific huge page setup. It can even be called later >>> at the time of hugetlb command line processing. >> >> Yes, now that hugetlb_cma_reserve() has been moved into hugetlb_add_hstate(). >> But then there is an explicit warning while trying to mix both the command >> line options i.e hugepagesz= and hugetlb_cma=. The proposed code here have >> not changed that behavior and hence the following warning should have been >> triggered here as well. >> >> 1) hugepagesz_setup() >> hugetlb_add_hstate() >> hugetlb_cma_reserve() >> >> 2) hugepages_setup() >> hugetlb_hstate_alloc_pages() when order >= MAX_ORDER >> >> if (hstate_is_gigantic(h)) { >> if (IS_ENABLED(CONFIG_CMA) && hugetlb_cma[0]) { >> pr_warn_once("HugeTLB: hugetlb_cma is enabled, skip boot time allocation\n"); >> break; >> } >> if (!alloc_bootmem_huge_page(h)) >> break; >> } >> >> Nonetheless, it does not make sense to mix both memblock and CMA based huge >> page pre-allocations. But looking at this again, could this warning be ever >> triggered till now ? Unless, a given platform calls hugetlb_cma_reserve() >> before _setup("hugepages=", hugepages_setup). Anyways, there seems to be >> good reasons to keep both memblock and CMA based pre-allocations in place. >> But mixing them together (as done in the proposed code here) does not seem >> to be right. > > I'm not sure if I follow the question. > > This proposal does not change the trigger for the warning printed when one > tries to both reserve CMA and pre-allocate gigantic pages. If hugetlb_cma > is specified on the command line, and someone tries to pre-allocate gigantic > pages they will get the warning. Such a command line on x86 might look like, > hugetlb_cma=4G hugepagesz=1G hugepages=4 > > You will then see, > [ 0.065864] HugeTLB: hugetlb_cma is enabled, skip boot time allocation > [ 0.065866] HugeTLB: allocating 4 of page size 1.00 GiB failed. Only allocated 0 hugepages. > > Ideally we could/should eliminate the second message. > This behavior exists in the current code. There is variant of this which is pseries powerpc where there is hypervisor assistance w.r.t allocating gigantic pages. See the ppc64 enablement patch https://lore.kernel.org/linuxppc-dev/20200713150749.25245-1-aneesh.kumar@linux.ibm.com/ -aneesh _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel