From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B2D02C433EF for ; Fri, 17 Sep 2021 06:26:25 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7458A60FC2 for ; Fri, 17 Sep 2021 06:26:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 7458A60FC2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linuxfoundation.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=9GMZIJHkQD9Qk9pptOx/+X+uIrzA527Q2UMQtx8JCBQ=; b=ClBrSXcpMoqR6W beaGQIh2KcF8mIizn+C3eSto1goaEENoUBYyRIHRwT3kWkrbAOSOqI8BLX0GhalRyzuWKuQYe5oi+ 4w6pPOGjLKPtnD5uTEgz00SYESEld3qHWPzsYxEmB9DiuCse40na5WydV6iLkUcAxhiFU5WyDUfpT hN5nDVyRg0xWASsQXUbQm0qmJFmRuZSnROAY52w/3UrtqIfjvaSzAdfYuyaiqNJNngLTHG3ox11A7 D4wPqkW8k/+uBHkM+iZ7SylI6PHcS/b0TIYdEGFvjix1T1xZHtYYkRZ3GGQGBFNbn5qSRhOnicq0r 2Cuae513Njvd60HUCuhQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mR7IT-00DIaG-Dj; Fri, 17 Sep 2021 06:24:29 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mR7IP-00DIZX-FS for linux-arm-kernel@lists.infradead.org; Fri, 17 Sep 2021 06:24:26 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 4F37260F4A; Fri, 17 Sep 2021 06:24:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1631859864; bh=2Yf48r3eL6AsFDGAGrmnb1zCRH5QI/gs+4Dz7P0jxvc=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=tNzW5LF8aK9JS7ekdhbBuxILwkGT8IsK1tdD6XVWfaxerpka4xEbxOGQfx9TmYCbc cxDXJDPgxFELIdlXNagFqAUyC5vFlnalOBarNwHj2aJLgeX+Q8vbwzDAzASgT6bTpi Aop6e1159Fq4chjUaevnB9GrIwlAnoUtolhMnHHg= Date: Fri, 17 Sep 2021 08:24:22 +0200 From: Greg KH To: Kefeng Wang Cc: will@kernel.org, catalin.marinas@arm.com, ryabinin.a.a@gmail.com, andreyknvl@gmail.com, dvyukov@google.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, elver@google.com, akpm@linux-foundation.org, kasan-dev@googlegroups.com Subject: Re: [PATCH v4 2/3] arm64: Support page mapping percpu first chunk allocator Message-ID: References: <20210910053354.26721-1-wangkefeng.wang@huawei.com> <20210910053354.26721-3-wangkefeng.wang@huawei.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20210910053354.26721-3-wangkefeng.wang@huawei.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210916_232425_597861_4F5500EA X-CRM114-Status: GOOD ( 25.69 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, Sep 10, 2021 at 01:33:53PM +0800, Kefeng Wang wrote: > Percpu embedded first chunk allocator is the firstly option, but it > could fails on ARM64, eg, > "percpu: max_distance=0x5fcfdc640000 too large for vmalloc space 0x781fefff0000" > "percpu: max_distance=0x600000540000 too large for vmalloc space 0x7dffb7ff0000" > "percpu: max_distance=0x5fff9adb0000 too large for vmalloc space 0x5dffb7ff0000" > > then we could meet "WARNING: CPU: 15 PID: 461 at vmalloc.c:3087 pcpu_get_vm_areas+0x488/0x838", > even the system could not boot successfully. > > Let's implement page mapping percpu first chunk allocator as a fallback > to the embedding allocator to increase the robustness of the system. > > Reviewed-by: Catalin Marinas > Signed-off-by: Kefeng Wang > --- > arch/arm64/Kconfig | 4 ++ > drivers/base/arch_numa.c | 82 +++++++++++++++++++++++++++++++++++----- > 2 files changed, 76 insertions(+), 10 deletions(-) > > diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig > index 077f2ec4eeb2..04cfe1b4e98b 100644 > --- a/arch/arm64/Kconfig > +++ b/arch/arm64/Kconfig > @@ -1042,6 +1042,10 @@ config NEED_PER_CPU_EMBED_FIRST_CHUNK > def_bool y > depends on NUMA > > +config NEED_PER_CPU_PAGE_FIRST_CHUNK > + def_bool y > + depends on NUMA Why is this a config option at all? > + > source "kernel/Kconfig.hz" > > config ARCH_SPARSEMEM_ENABLE > diff --git a/drivers/base/arch_numa.c b/drivers/base/arch_numa.c > index 46c503486e96..995dca9f3254 100644 > --- a/drivers/base/arch_numa.c > +++ b/drivers/base/arch_numa.c > @@ -14,6 +14,7 @@ > #include > > #include > +#include > > struct pglist_data *node_data[MAX_NUMNODES] __read_mostly; > EXPORT_SYMBOL(node_data); > @@ -168,22 +169,83 @@ static void __init pcpu_fc_free(void *ptr, size_t size) > memblock_free_early(__pa(ptr), size); > } > > +#ifdef CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK Ick, no #ifdef in .c files if at all possible please. > +static void __init pcpu_populate_pte(unsigned long addr) > +{ > + pgd_t *pgd = pgd_offset_k(addr); > + p4d_t *p4d; > + pud_t *pud; > + pmd_t *pmd; > + > + p4d = p4d_offset(pgd, addr); > + if (p4d_none(*p4d)) { > + pud_t *new; > + > + new = memblock_alloc(PAGE_SIZE, PAGE_SIZE); > + if (!new) > + goto err_alloc; > + p4d_populate(&init_mm, p4d, new); > + } > + > + pud = pud_offset(p4d, addr); > + if (pud_none(*pud)) { > + pmd_t *new; > + > + new = memblock_alloc(PAGE_SIZE, PAGE_SIZE); > + if (!new) > + goto err_alloc; > + pud_populate(&init_mm, pud, new); > + } > + > + pmd = pmd_offset(pud, addr); > + if (!pmd_present(*pmd)) { > + pte_t *new; > + > + new = memblock_alloc(PAGE_SIZE, PAGE_SIZE); > + if (!new) > + goto err_alloc; > + pmd_populate_kernel(&init_mm, pmd, new); > + } > + > + return; > + > +err_alloc: > + panic("%s: Failed to allocate %lu bytes align=%lx from=%lx\n", > + __func__, PAGE_SIZE, PAGE_SIZE, PAGE_SIZE); That feels harsh, are you sure you want to crash? There's no way to recover from this? If not, how can this fail in real life? > +} > +#endif > + > void __init setup_per_cpu_areas(void) > { > unsigned long delta; > unsigned int cpu; > - int rc; > + int rc = -EINVAL; > + > + if (pcpu_chosen_fc != PCPU_FC_PAGE) { > + /* > + * Always reserve area for module percpu variables. That's > + * what the legacy allocator did. > + */ > + rc = pcpu_embed_first_chunk(PERCPU_MODULE_RESERVE, > + PERCPU_DYNAMIC_RESERVE, PAGE_SIZE, > + pcpu_cpu_distance, > + pcpu_fc_alloc, pcpu_fc_free); > +#ifdef CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK > + if (rc < 0) > + pr_warn("PERCPU: %s allocator failed (%d), falling back to page size\n", > + pcpu_fc_names[pcpu_chosen_fc], rc); > +#endif Why only print out a message for a config option? Again, no #ifdef in .c files if at all possible. thanks, greg k-h _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9ABB8C433F5 for ; Fri, 17 Sep 2021 06:24:27 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1D8326103C for ; Fri, 17 Sep 2021 06:24:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 1D8326103C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linuxfoundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id A98D1900002; Fri, 17 Sep 2021 02:24:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A48C96B0072; Fri, 17 Sep 2021 02:24:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 93723900002; Fri, 17 Sep 2021 02:24:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0212.hostedemail.com [216.40.44.212]) by kanga.kvack.org (Postfix) with ESMTP id 83D646B0071 for ; Fri, 17 Sep 2021 02:24:26 -0400 (EDT) Received: from smtpin40.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 367EA1845142C for ; Fri, 17 Sep 2021 06:24:26 +0000 (UTC) X-FDA: 78596076132.40.E5F4C21 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf19.hostedemail.com (Postfix) with ESMTP id C9735B0000A5 for ; Fri, 17 Sep 2021 06:24:25 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 4F37260F4A; Fri, 17 Sep 2021 06:24:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1631859864; bh=2Yf48r3eL6AsFDGAGrmnb1zCRH5QI/gs+4Dz7P0jxvc=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=tNzW5LF8aK9JS7ekdhbBuxILwkGT8IsK1tdD6XVWfaxerpka4xEbxOGQfx9TmYCbc cxDXJDPgxFELIdlXNagFqAUyC5vFlnalOBarNwHj2aJLgeX+Q8vbwzDAzASgT6bTpi Aop6e1159Fq4chjUaevnB9GrIwlAnoUtolhMnHHg= Date: Fri, 17 Sep 2021 08:24:22 +0200 From: Greg KH To: Kefeng Wang Cc: will@kernel.org, catalin.marinas@arm.com, ryabinin.a.a@gmail.com, andreyknvl@gmail.com, dvyukov@google.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, elver@google.com, akpm@linux-foundation.org, kasan-dev@googlegroups.com Subject: Re: [PATCH v4 2/3] arm64: Support page mapping percpu first chunk allocator Message-ID: References: <20210910053354.26721-1-wangkefeng.wang@huawei.com> <20210910053354.26721-3-wangkefeng.wang@huawei.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20210910053354.26721-3-wangkefeng.wang@huawei.com> X-Stat-Signature: 1bktas3ozprd4ui5r8d11mk8gfikh9dx Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=linuxfoundation.org header.s=korg header.b=tNzW5LF8; dmarc=pass (policy=none) header.from=linuxfoundation.org; spf=pass (imf19.hostedemail.com: domain of gregkh@linuxfoundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=gregkh@linuxfoundation.org X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: C9735B0000A5 X-HE-Tag: 1631859865-228544 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Sep 10, 2021 at 01:33:53PM +0800, Kefeng Wang wrote: > Percpu embedded first chunk allocator is the firstly option, but it > could fails on ARM64, eg, > "percpu: max_distance=0x5fcfdc640000 too large for vmalloc space 0x781fefff0000" > "percpu: max_distance=0x600000540000 too large for vmalloc space 0x7dffb7ff0000" > "percpu: max_distance=0x5fff9adb0000 too large for vmalloc space 0x5dffb7ff0000" > > then we could meet "WARNING: CPU: 15 PID: 461 at vmalloc.c:3087 pcpu_get_vm_areas+0x488/0x838", > even the system could not boot successfully. > > Let's implement page mapping percpu first chunk allocator as a fallback > to the embedding allocator to increase the robustness of the system. > > Reviewed-by: Catalin Marinas > Signed-off-by: Kefeng Wang > --- > arch/arm64/Kconfig | 4 ++ > drivers/base/arch_numa.c | 82 +++++++++++++++++++++++++++++++++++----- > 2 files changed, 76 insertions(+), 10 deletions(-) > > diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig > index 077f2ec4eeb2..04cfe1b4e98b 100644 > --- a/arch/arm64/Kconfig > +++ b/arch/arm64/Kconfig > @@ -1042,6 +1042,10 @@ config NEED_PER_CPU_EMBED_FIRST_CHUNK > def_bool y > depends on NUMA > > +config NEED_PER_CPU_PAGE_FIRST_CHUNK > + def_bool y > + depends on NUMA Why is this a config option at all? > + > source "kernel/Kconfig.hz" > > config ARCH_SPARSEMEM_ENABLE > diff --git a/drivers/base/arch_numa.c b/drivers/base/arch_numa.c > index 46c503486e96..995dca9f3254 100644 > --- a/drivers/base/arch_numa.c > +++ b/drivers/base/arch_numa.c > @@ -14,6 +14,7 @@ > #include > > #include > +#include > > struct pglist_data *node_data[MAX_NUMNODES] __read_mostly; > EXPORT_SYMBOL(node_data); > @@ -168,22 +169,83 @@ static void __init pcpu_fc_free(void *ptr, size_t size) > memblock_free_early(__pa(ptr), size); > } > > +#ifdef CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK Ick, no #ifdef in .c files if at all possible please. > +static void __init pcpu_populate_pte(unsigned long addr) > +{ > + pgd_t *pgd = pgd_offset_k(addr); > + p4d_t *p4d; > + pud_t *pud; > + pmd_t *pmd; > + > + p4d = p4d_offset(pgd, addr); > + if (p4d_none(*p4d)) { > + pud_t *new; > + > + new = memblock_alloc(PAGE_SIZE, PAGE_SIZE); > + if (!new) > + goto err_alloc; > + p4d_populate(&init_mm, p4d, new); > + } > + > + pud = pud_offset(p4d, addr); > + if (pud_none(*pud)) { > + pmd_t *new; > + > + new = memblock_alloc(PAGE_SIZE, PAGE_SIZE); > + if (!new) > + goto err_alloc; > + pud_populate(&init_mm, pud, new); > + } > + > + pmd = pmd_offset(pud, addr); > + if (!pmd_present(*pmd)) { > + pte_t *new; > + > + new = memblock_alloc(PAGE_SIZE, PAGE_SIZE); > + if (!new) > + goto err_alloc; > + pmd_populate_kernel(&init_mm, pmd, new); > + } > + > + return; > + > +err_alloc: > + panic("%s: Failed to allocate %lu bytes align=%lx from=%lx\n", > + __func__, PAGE_SIZE, PAGE_SIZE, PAGE_SIZE); That feels harsh, are you sure you want to crash? There's no way to recover from this? If not, how can this fail in real life? > +} > +#endif > + > void __init setup_per_cpu_areas(void) > { > unsigned long delta; > unsigned int cpu; > - int rc; > + int rc = -EINVAL; > + > + if (pcpu_chosen_fc != PCPU_FC_PAGE) { > + /* > + * Always reserve area for module percpu variables. That's > + * what the legacy allocator did. > + */ > + rc = pcpu_embed_first_chunk(PERCPU_MODULE_RESERVE, > + PERCPU_DYNAMIC_RESERVE, PAGE_SIZE, > + pcpu_cpu_distance, > + pcpu_fc_alloc, pcpu_fc_free); > +#ifdef CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK > + if (rc < 0) > + pr_warn("PERCPU: %s allocator failed (%d), falling back to page size\n", > + pcpu_fc_names[pcpu_chosen_fc], rc); > +#endif Why only print out a message for a config option? Again, no #ifdef in .c files if at all possible. thanks, greg k-h