From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EBE14C433FE for ; Mon, 7 Dec 2020 07:31:51 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 87ED1221F8 for ; Mon, 7 Dec 2020 07:31:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 87ED1221F8 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From: References:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=uHxvGV/de2UctHg8ykcx5KBDQqTCjAtTsxF3HFVac+8=; b=NOfbTkxCcxg5Oilc75U4ErdyX szNgO2rWwspQCzaYQEh8+qv7oHMVz/noUfjUjt3SIp9GVm6wPVnhq7xJjak8WIg/w0fbaW2J4pyxi CN6jde+/aEM1HC/pQ54BMpmXTNmzQIDq/Ph7R3kulGonC/k7IS9DAmr4nZaIct4OJgFMqi0YImO2K WySqFNIOIB+WTy6dRDBLWAMdN5mhjn8CECMwYu6kOAwgldKzWINP9GrrxIB9+fNNpMy12YpyQB/Sy LxTiRaSztPe3zVGM8WFS3XEVysup9POGh+8YE1DJs1sCotEMtEkDI+HTAMO0am98NgC9txf20cc6K Vl2O/a8iA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kmAyo-000533-EP; Mon, 07 Dec 2020 07:30:42 +0000 Received: from foss.arm.com ([217.140.110.172]) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kmAym-00052d-Es for linux-arm-kernel@lists.infradead.org; Mon, 07 Dec 2020 07:30:41 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2CCA511D4; Sun, 6 Dec 2020 23:30:35 -0800 (PST) Received: from [10.163.86.92] (unknown [10.163.86.92]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 27B623F66B; Sun, 6 Dec 2020 23:30:31 -0800 (PST) Subject: Re: [PATCH] arm64: mm: decrease the section size to reduce the memory reserved for the page map To: "Song Bao Hua (Barry Song)" , Mike Rapoport , Will Deacon References: <20201204014443.43329-1-liwei213@huawei.com> <20201204111347.GA844@willie-the-truck> <20201204114400.GT123287@linux.ibm.com> <60cb36d5dfcb4f9c904a83b520ecfe84@hisilicon.com> From: Anshuman Khandual Message-ID: Date: Mon, 7 Dec 2020 13:00:33 +0530 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: <60cb36d5dfcb4f9c904a83b520ecfe84@hisilicon.com> Content-Language: en-US X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201207_023040_601459_B1959FC7 X-CRM114-Status: GOOD ( 24.21 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "linux-arm-kernel@lists.infradead.org" , "steve.capper@arm.com" , "catalin.marinas@arm.com" , "linux-kernel@vger.kernel.org" , "liwei \(CM\)" , fengbaopeng , "nsaenzjulienne@suse.de" , butao Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 12/7/20 7:10 AM, Song Bao Hua (Barry Song) wrote: > > >> -----Original Message----- >> From: Mike Rapoport [mailto:rppt@linux.ibm.com] >> Sent: Saturday, December 5, 2020 12:44 AM >> To: Will Deacon >> Cc: liwei (CM) ; catalin.marinas@arm.com; fengbaopeng >> ; nsaenzjulienne@suse.de; steve.capper@arm.com; >> Song Bao Hua (Barry Song) ; >> linux-arm-kernel@lists.infradead.org; linux-kernel@vger.kernel.org; butao >> >> Subject: Re: [PATCH] arm64: mm: decrease the section size to reduce the memory >> reserved for the page map >> >> On Fri, Dec 04, 2020 at 11:13:47AM +0000, Will Deacon wrote: >>> On Fri, Dec 04, 2020 at 09:44:43AM +0800, Wei Li wrote: >>>> For the memory hole, sparse memory model that define SPARSEMEM_VMEMMAP >>>> do not free the reserved memory for the page map, decrease the section >>>> size can reduce the waste of reserved memory. >>>> >>>> Signed-off-by: Wei Li >>>> Signed-off-by: Baopeng Feng >>>> Signed-off-by: Xia Qing >>>> --- >>>> arch/arm64/include/asm/sparsemem.h | 2 +- >>>> 1 file changed, 1 insertion(+), 1 deletion(-) >>>> >>>> diff --git a/arch/arm64/include/asm/sparsemem.h >> b/arch/arm64/include/asm/sparsemem.h >>>> index 1f43fcc79738..8963bd3def28 100644 >>>> --- a/arch/arm64/include/asm/sparsemem.h >>>> +++ b/arch/arm64/include/asm/sparsemem.h >>>> @@ -7,7 +7,7 @@ >>>> >>>> #ifdef CONFIG_SPARSEMEM >>>> #define MAX_PHYSMEM_BITS CONFIG_ARM64_PA_BITS >>>> -#define SECTION_SIZE_BITS 30 >>>> +#define SECTION_SIZE_BITS 27 >>> >>> We chose '30' to avoid running out of bits in the page flags. What changed? >> >> I think that for 64-bit there are still plenty of free bits. I didn't >> check now, but when I played with SPARSEMEM on m68k there were 8 bits >> for section out of 32. >> >>> With this patch, I can trigger: >>> >>> ./include/linux/mmzone.h:1170:2: error: Allocator MAX_ORDER exceeds >> SECTION_SIZE >>> #error Allocator MAX_ORDER exceeds SECTION_SIZE >>> >>> if I bump up NR_CPUS and NODES_SHIFT. >> >> I don't think it's related to NR_CPUS and NODES_SHIFT. >> This seems rather 64K pages that cause this. >> >> Not that is shouldn't be addressed. > > Right now, only 4K PAGES will define ARM64_SWAPPER_USES_SECTION_MAPS. > Other cases will use vmemmap_populate_basepages(). > The original patch should be only addressing the issue in 4K pages: > https://lore.kernel.org/lkml/20200812010655.96339-1-liwei213@huawei.com/ > > would we do something like the below? > #ifdef CONFIG_ARM64_4K_PAGE > #define SECTION_SIZE_BITS 27 > #else > #define SECTION_SIZE_BITS 30 > #endif This is bit arbitrary. Probably 27 can be further reduced for 4K page size. Instead, we should make SECTION_SIZE_BITS explicitly depend upon MAX_ORDER. IOW section size should be the same as the highest order page in the buddy. CONFIG_FORCE_MAX_ZONEORDER is always defined on arm64. A quick test shows SECTION_SIZE_BITS would be 22 on 4K pages and 29 for 64K pages. As a fall back SECTION_SIZE_BITS can still be 30 in case CONFIG_FORCE_MAX_ZONEORDER is not defined. --- a/arch/arm64/include/asm/sparsemem.h +++ b/arch/arm64/include/asm/sparsemem.h @@ -7,7 +7,7 @@ #ifdef CONFIG_SPARSEMEM #define MAX_PHYSMEM_BITS CONFIG_ARM64_PA_BITS -#define SECTION_SIZE_BITS 30 +#define SECTION_SIZE_BITS (CONFIG_FORCE_MAX_ZONEORDER - 1 + PAGE_SHIFT) #endif #endif A similar approach exists on ia64 platform as well. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.2 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3D89BC433FE for ; Mon, 7 Dec 2020 07:31:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 01CEC221F8 for ; Mon, 7 Dec 2020 07:31:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726126AbgLGHbV (ORCPT ); Mon, 7 Dec 2020 02:31:21 -0500 Received: from foss.arm.com ([217.140.110.172]:42858 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725783AbgLGHbV (ORCPT ); Mon, 7 Dec 2020 02:31:21 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2CCA511D4; Sun, 6 Dec 2020 23:30:35 -0800 (PST) Received: from [10.163.86.92] (unknown [10.163.86.92]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 27B623F66B; Sun, 6 Dec 2020 23:30:31 -0800 (PST) Subject: Re: [PATCH] arm64: mm: decrease the section size to reduce the memory reserved for the page map To: "Song Bao Hua (Barry Song)" , Mike Rapoport , Will Deacon Cc: "steve.capper@arm.com" , "catalin.marinas@arm.com" , "linux-kernel@vger.kernel.org" , "nsaenzjulienne@suse.de" , "liwei (CM)" , butao , "linux-arm-kernel@lists.infradead.org" , fengbaopeng References: <20201204014443.43329-1-liwei213@huawei.com> <20201204111347.GA844@willie-the-truck> <20201204114400.GT123287@linux.ibm.com> <60cb36d5dfcb4f9c904a83b520ecfe84@hisilicon.com> From: Anshuman Khandual Message-ID: Date: Mon, 7 Dec 2020 13:00:33 +0530 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: <60cb36d5dfcb4f9c904a83b520ecfe84@hisilicon.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 12/7/20 7:10 AM, Song Bao Hua (Barry Song) wrote: > > >> -----Original Message----- >> From: Mike Rapoport [mailto:rppt@linux.ibm.com] >> Sent: Saturday, December 5, 2020 12:44 AM >> To: Will Deacon >> Cc: liwei (CM) ; catalin.marinas@arm.com; fengbaopeng >> ; nsaenzjulienne@suse.de; steve.capper@arm.com; >> Song Bao Hua (Barry Song) ; >> linux-arm-kernel@lists.infradead.org; linux-kernel@vger.kernel.org; butao >> >> Subject: Re: [PATCH] arm64: mm: decrease the section size to reduce the memory >> reserved for the page map >> >> On Fri, Dec 04, 2020 at 11:13:47AM +0000, Will Deacon wrote: >>> On Fri, Dec 04, 2020 at 09:44:43AM +0800, Wei Li wrote: >>>> For the memory hole, sparse memory model that define SPARSEMEM_VMEMMAP >>>> do not free the reserved memory for the page map, decrease the section >>>> size can reduce the waste of reserved memory. >>>> >>>> Signed-off-by: Wei Li >>>> Signed-off-by: Baopeng Feng >>>> Signed-off-by: Xia Qing >>>> --- >>>> arch/arm64/include/asm/sparsemem.h | 2 +- >>>> 1 file changed, 1 insertion(+), 1 deletion(-) >>>> >>>> diff --git a/arch/arm64/include/asm/sparsemem.h >> b/arch/arm64/include/asm/sparsemem.h >>>> index 1f43fcc79738..8963bd3def28 100644 >>>> --- a/arch/arm64/include/asm/sparsemem.h >>>> +++ b/arch/arm64/include/asm/sparsemem.h >>>> @@ -7,7 +7,7 @@ >>>> >>>> #ifdef CONFIG_SPARSEMEM >>>> #define MAX_PHYSMEM_BITS CONFIG_ARM64_PA_BITS >>>> -#define SECTION_SIZE_BITS 30 >>>> +#define SECTION_SIZE_BITS 27 >>> >>> We chose '30' to avoid running out of bits in the page flags. What changed? >> >> I think that for 64-bit there are still plenty of free bits. I didn't >> check now, but when I played with SPARSEMEM on m68k there were 8 bits >> for section out of 32. >> >>> With this patch, I can trigger: >>> >>> ./include/linux/mmzone.h:1170:2: error: Allocator MAX_ORDER exceeds >> SECTION_SIZE >>> #error Allocator MAX_ORDER exceeds SECTION_SIZE >>> >>> if I bump up NR_CPUS and NODES_SHIFT. >> >> I don't think it's related to NR_CPUS and NODES_SHIFT. >> This seems rather 64K pages that cause this. >> >> Not that is shouldn't be addressed. > > Right now, only 4K PAGES will define ARM64_SWAPPER_USES_SECTION_MAPS. > Other cases will use vmemmap_populate_basepages(). > The original patch should be only addressing the issue in 4K pages: > https://lore.kernel.org/lkml/20200812010655.96339-1-liwei213@huawei.com/ > > would we do something like the below? > #ifdef CONFIG_ARM64_4K_PAGE > #define SECTION_SIZE_BITS 27 > #else > #define SECTION_SIZE_BITS 30 > #endif This is bit arbitrary. Probably 27 can be further reduced for 4K page size. Instead, we should make SECTION_SIZE_BITS explicitly depend upon MAX_ORDER. IOW section size should be the same as the highest order page in the buddy. CONFIG_FORCE_MAX_ZONEORDER is always defined on arm64. A quick test shows SECTION_SIZE_BITS would be 22 on 4K pages and 29 for 64K pages. As a fall back SECTION_SIZE_BITS can still be 30 in case CONFIG_FORCE_MAX_ZONEORDER is not defined. --- a/arch/arm64/include/asm/sparsemem.h +++ b/arch/arm64/include/asm/sparsemem.h @@ -7,7 +7,7 @@ #ifdef CONFIG_SPARSEMEM #define MAX_PHYSMEM_BITS CONFIG_ARM64_PA_BITS -#define SECTION_SIZE_BITS 30 +#define SECTION_SIZE_BITS (CONFIG_FORCE_MAX_ZONEORDER - 1 + PAGE_SHIFT) #endif #endif A similar approach exists on ia64 platform as well.