From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 120E5C47077 for ; Wed, 17 Jan 2024 03:17:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 859586B00B6; Tue, 16 Jan 2024 22:17:32 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 809456B00B8; Tue, 16 Jan 2024 22:17:32 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6F7CB6B00B9; Tue, 16 Jan 2024 22:17:32 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 5F0FA6B00B6 for ; Tue, 16 Jan 2024 22:17:32 -0500 (EST) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 2095A80733 for ; Wed, 17 Jan 2024 03:17:32 +0000 (UTC) X-FDA: 81687342744.13.196DF5A Received: from out-171.mta1.migadu.com (out-171.mta1.migadu.com [95.215.58.171]) by imf28.hostedemail.com (Postfix) with ESMTP id 14C08C0004 for ; Wed, 17 Jan 2024 03:17:29 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=BD7Wwthn; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf28.hostedemail.com: domain of yajun.deng@linux.dev designates 95.215.58.171 as permitted sender) smtp.mailfrom=yajun.deng@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1705461450; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=XSC908aiFP9G9pfXZ6v8h/smDGh99yJWYPnGkGsZUbM=; b=utyrwjb8ZdwktVCWgLjb52HO4EbEdXUDiAdj9VbbX3c+UPaYgpEN10Etn7TYII+6Tyyidl xfM8TOOg2n1VNeiuRjGazgbDcpplHiJBpAGgmgeKRF0F/XUC0fvJI9snxZTBQWAhV3WyGR w11BaMw67o3Pxv/yEKQLRhtfnrUg+K0= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=BD7Wwthn; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf28.hostedemail.com: domain of yajun.deng@linux.dev designates 95.215.58.171 as permitted sender) smtp.mailfrom=yajun.deng@linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1705461450; a=rsa-sha256; cv=none; b=PXSgFNH07Um2oJV0tslBVbt2D6Jxe3tZKa0UqnEetT8DuM4oMfP041F0yxd9Hy1dKwaSOM 1QrFwGnSb4tz9xH41xFMle068qhy59Fsza5yklyXRGNJjEzMNDG9CPP5cMZ5O63l20q8WM qvdIHGvFWHyYUxq58eVDUerziRZcOp4= Message-ID: <234ddaeb-5988-858d-6ba1-0fef90fb7a87@linux.dev> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1705461447; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XSC908aiFP9G9pfXZ6v8h/smDGh99yJWYPnGkGsZUbM=; b=BD7WwthnxzeJ3LrTLqB9B0zYUZSWiYUzNYXW8d+KQ5RM+87cTCqdCGye32yPu6YAx48O4P SkztbWYZdaBh8JqcST9ndNHIQjTDKcuwbSPQP6RjU+B/FQIGzNMgH4kmmp6z7b/3yl2JP2 hyo4OXFnaVOs1Dg2BKlgYJ5eOmbQHE4= Date: Wed, 17 Jan 2024 11:17:18 +0800 MIME-Version: 1.0 Subject: Re: memblock_reserve for unadded region (was: [PATCH] MIPS: loongson64: fix boot failure) Content-Language: en-US To: Huang Pei Cc: Mike Rapoport , Jiaxun Yang , linux-mm@kvack.org, Bibo Mao , linux-mips@vger.kernel.org, Paul Burton , Li Xuefeng , Yang Tiezhu , Gao Juxin , Huacai Chen , Thomas Bogendoerfer , linux-kernel@vger.kernel.org References: <20231225093025.23215-1-huangpei@loongson.cn> <731134fd-4b3d-418c-84ee-80646bffcc01@flygoat.com> <20240116122304.qwzy7san2vgspt2x@Board-3A3000> <3fc2f75e-d163-1ad1-009a-0e4538011885@linux.dev> <20240117030113.gs2fjs6vydthsc6l@Board-3A3000> X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Yajun Deng In-Reply-To: <20240117030113.gs2fjs6vydthsc6l@Board-3A3000> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Migadu-Flow: FLOW_OUT X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 14C08C0004 X-Stat-Signature: e6f7nhqwf9emgcnf1k5qtrrktm56bkd7 X-HE-Tag: 1705461449-277229 X-HE-Meta: U2FsdGVkX1999pSnVHeR2QL9N1OCOHwg6MPctKnepc/1RTdwORM5y+vn2b5nWtK4/7xDnPkICdxYZ37aC+mBK4RpbeOgsbLNl+HjJmbEln/Gg0rKcSebdfg6GNGV2FMljlKOCACUixoTjcD3x68VrS33ItIuEuqIt8h1v39d1U25Mr6RmUcExWUwURWhHTG6EBxSHQRONeRd40tuj5TT6Tp25OJaGWLqYZVyiGZu2sHrXNZ4ureEVMJcP4EP4m1DATN4S8HlN6W/5K/M/H+8Ijg8T8/u+zeawCx9bgTZzYOW5T4KSkszG9gPP17l4ry02QJtt+ob9FM7O0zcbYm/jkGnD4HBR47/uH+KG5wvlIzTKlIg24HtlpMLHSS11fOnp6Gm7xDwXMJe8Ypeu731xDpruNJL0t11cy3Mg9GN0WGWiomQk1dJnG5kaTFJzGoSlft2VApn7oMw1CD26Iz0aE1RlW3Avz8+9Of+Is4AH+FTB6OfI7yncr2H5Pu0b5fbTgvp0mSSAZj3103DPeTYM2eb9RLmzA5vwqak0RzjortrWz8YXtlIr6t0gXzdCcAbMtMzrRMHXr9RDoJkMRRXrRpY16NhkDZKgZYrkd8ZkI3nFn4lBcGh0YEkzIUXNgcUTXnY/gA1DBJzr7dppbPwZwyN0K5T1Kt/61GXjtenssE18pph8zbXUPCe8G+GysUiF4oTOEU9JqzwJwwXxhzuo1wCCp+11N4j+/jE7z4eZEPOz0SQhVR0ryg/PlWY4YvxH64qB+mRqcxQYbhcqAiWwokpVKAYkgs3CvWrIs+xsrCJnrEOfXHNgWv4vD8JjJBSsN9Eij+siB46OAsnYDjkXpLOT3zIR2k91T6ak2fBb7kfCE7QDgy+Vinii1tUDlhbODxLAW1QFx050C6epf0jMb3y2N504cwPgG9inFgIEZo+a8bvyG+vjK+jdDWLTiIZ5amhNM1Vmb2AUnsbZdD e8l3gzhh 4dhwpwqqzhMe3loJE/CNgReSki908FV8YLqis/53QSx6vUFmwXgkUT8OBvEcyJdJV2w1Bu+o3z7lskZjcrW/E+tUg+UrHe9pBlINUJ1I2q6ftJtdKB+YK07wrKqZxIEFkCt6HjxOrZQ1Umw0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2024/1/17 11:01, Huang Pei wrote: > On Wed, Jan 17, 2024 at 10:20:00AM +0800, Yajun Deng wrote: >> On 2024/1/16 20:23, Huang Pei wrote: >>> On Tue, Jan 16, 2024 at 10:39:04AM +0200, Mike Rapoport wrote: >>>> On Mon, Jan 15, 2024 at 02:08:21PM +0000, Jiaxun Yang wrote: >>>>> Hi mm folks, >>>>> >>>>> Just a quick question, what is the expected behavior of memblock_reserve >>>>> a region that is not added to memblock with memblock_add? >>>>> >>>>> I'm unable to find any documentation about memblock_reserve in comments and >>>>> boot-time-mm, but as per my understanding to the code, this should be a >>>>> legit usage? >>>> Yes, memblock allows reserving memory that was not added to memblock with >>>> memblock_add(). >>> I think arch/platform specific code should fix this bug, like, >>> -------------------------------------------------------------------------- >>> //for loongson64 >>> memblock_set_node(0, 1ULL << 44, &memblock.reserved, 0); >>> >>> -------------------------------------------------------------------------- >>> >>> or maybe memblock provide something like memblock_reserve_node >> Hi pei, >> >> Can you test the following patch to see if it fixes this bug? >> >> diff --git a/mm/mm_init.c b/mm/mm_init.c >> index 2c19f5515e36..97721d99fdce 100644 >> --- a/mm/mm_init.c >> +++ b/mm/mm_init.c >> @@ -708,6 +708,9 @@ static void __meminit init_reserved_page(unsigned long >> pfn, int nid) >>         pg_data_t *pgdat; >>         int zid; >> >> +       if (unlikely(nid == NUMA_NO_NODE || nid >= MAX_NUMNODES)) >> +               nid = early_pfn_to_nid(pfn); >> + >>         if (early_page_initialised(pfn, nid)) >>                 return; > I do not think this fix set the right nid, ONLY arch/platform know that > nid > > int __meminit early_pfn_to_nid(unsigned long pfn) > { > static DEFINE_SPINLOCK(early_pfn_lock); > int nid; > > spin_lock(&early_pfn_lock); > nid = __early_pfn_to_nid(pfn, > &early_pfnnid_cache); > if (nid < 0) > //!!!first_online_node MAY NOT be the node the pfn belong to!!! > nid = first_online_node; > > spin_unlock(&early_pfn_lock); > > return > nid; > } Okay, I don't think this bug is caused by commit 61167ad5fecd ("mm: pass nid to reserve_bootmem_region()"), because even if you revert this commit, it will still get nid by early_pfn_to_nid(). Did I get that right? >> >>>>> In practical we run into uninitialized nid of reserved block problem, should >>>>> we fix it >>>>> in our usage, or on memblock side? >>>> Apparently it's a bug in memblock :( >>>> >>>> If you revert 61167ad5fecd ("mm: pass nid to reserve_bootmem_region()") >>>> does the issue disappear? >>> Yes, I git bisect this commit. >>> >>> But I don't think it is a bug in memblock. IMO, memblock_reserve under >>> NUMA set nid of reserved region to MAX_NUMNODES, which is the point >>> that cause the "memblock_get_region_node from memmap_init_reserved_pages " >>> passing a invalid node id(aka MAX_NUMNODES) to "reserver_bootmem_region >>> -> init_reserved_page -> early_pfn_to_nid". If arch-specific code DOES NOT >>> initialize the nid of reserved region(only it know that), or the reserved >>> region NOT within a memblock added by memblock_add, memblock can not >>> give a valid node id to the reserved region. Commit 61167ad5fecd ("mm: pass nid to >>> reserve_bootmem_region()") just reveals the embarrassment case by an >>> out of bound memory access. >>> >>>>> Thanks >>>>> >>>>> 在 2023/12/25 09:30, Huang Pei 写道: >>>>>> Since commit 61167ad5fecd("mm: pass nid to reserve_bootmem_region()), >>>>>> loongson64 booting failed with CONFIG_DEFERRED_STRUCT_PAGE_INIT like >>>>>> this: >>>>>> ---------------------------------------------------------------------- >>>>>> Call Trace: >>>>>> [] reserve_bootmem_region+0xa8/0x184 >>>>>> [] memblock_free_all+0x104/0x2a8 >>>>>> [] mem_init+0x84/0x94 >>>>>> [] mm_core_init+0xf8/0x308 >>>>>> [] start_kernel+0x43c/0x86c >>>>>> >>>>>> Code: 10400028 2402fff0 de420000 0203182b 14600022 >>>>>> 64420070 00003025 24040003 >>>>>> >>>>>> ---[ end trace 0000000000000000 ]--- >>>>>> Kernel panic - not syncing: Attempted to kill the idle task! >>>>>> ---[ end Kernel panic - not syncing: Attempted to kill the idle task! ]--- >>>>>> ---------------------------------------------------------------------- >>>>>> >>>>>> The root cause is no memory region "0x0-0x1fffff" paired with >>>>>> memory-reserved region "0x0-0x1fffff" and "0x0-0xfff", with "memblock >>>>>> =debug": >>>>>> >>>>>> ---------------------------------------------------------------------- >>>>>> memory[0x0] [0x0000000000200000-0x000000000effffff], >>>>>> 0x000000000ee00000 bytes on node 0 flags: 0x0 !!!!here >>>>>> memory[0x1] [0x0000000090000000-0x00000000fdffffff], >>>>>> 0x000000006e000000 bytes on node 0 flags: 0x0 >>>>>> memory[0x2] [0x0000000100000000-0x000000027fffffff], >>>>>> 0x0000000180000000 bytes on node 0 flags: 0x0 >>>>>> memory[0x3] [0x0000100000000000-0x000010000fffffff], >>>>>> 0x0000000010000000 bytes on node 1 flags: 0x0 >>>>>> memory[0x4] [0x0000100090000000-0x000010027fffffff], >>>>>> 0x00000001f0000000 bytes on node 1 flags: 0x0 >>>>>> reserved.cnt = 0x1f >>>>>> reserved[0x0] [0x0000000000000000-0x000000000190c80a], >>>>>> 0x000000000190c80b bytes flags: 0x0 !!!!oops 0x0-0x1fffff not in memory[0] >>>>>> reserved[0x1] [0x000000000190c810-0x000000000190eea3], >>>>>> 0x0000000000002694 bytes flags: 0x0 >>>>>> ---------------------------------------------------------------------- >>>>>> >>>>>> It caused memory-reserved region "0x0-0x1fffff" without valid node id >>>>>> in "memblock_get_region_node" from "memmap_init_reserved_pages", lead to >>>>>> "reserve_bootmem_region-> init_reserved_page -> early_pfn_to_nid()" >>>>>> accessing "node_data" out of bound. >>>>>> >>>>>> To fix this bug, we should remove unnecessary memory block reservation. >>>>>> >>>>>> +. no need to reserve 0x0-0x1fffff below kernel loading address, since >>>>>> it is not registered by "memblock_add_node" >>>>>> >>>>>> +. no need to reserve 0x0-0xfff for exception handling if it is not >>>>>> registered by "memblock_add" either. >>>>>> >>>>>> Fixes: commit 61167ad5fecd("mm: pass nid to reserve_bootmem_region()) >>>>>> Signed-off-by: Huang Pei >>>>>> --- >>>>>> arch/mips/kernel/traps.c | 3 ++- >>>>>> arch/mips/loongson64/numa.c | 2 -- >>>>>> 2 files changed, 2 insertions(+), 3 deletions(-) >>>>>> >>>>>> diff --git a/arch/mips/kernel/traps.c b/arch/mips/kernel/traps.c >>>>>> index 246c6a6b0261..9b632b4c10c3 100644 >>>>>> --- a/arch/mips/kernel/traps.c >>>>>> +++ b/arch/mips/kernel/traps.c >>>>>> @@ -2007,7 +2007,8 @@ unsigned long vi_handlers[64]; >>>>>> void reserve_exception_space(phys_addr_t addr, unsigned long size) >>>>>> { >>>>>> - memblock_reserve(addr, size); >>>>>> + if(memblock_is_region_memory(addr, size)) >>>>>> + memblock_reserve(addr, size); >>>>>> } >>>>>> void __init *set_except_vector(int n, void *addr) >>>>>> diff --git a/arch/mips/loongson64/numa.c b/arch/mips/loongson64/numa.c >>>>>> index 8f61e93c0c5b..0f516dde81da 100644 >>>>>> --- a/arch/mips/loongson64/numa.c >>>>>> +++ b/arch/mips/loongson64/numa.c >>>>>> @@ -130,8 +130,6 @@ static void __init node_mem_init(unsigned int node) >>>>>> memblock_reserve((node_addrspace_offset | 0xfe000000), >>>>>> 32 << 20); >>>>>> - /* Reserve pfn range 0~node[0]->node_start_pfn */ >>>>>> - memblock_reserve(0, PAGE_SIZE * start_pfn); >>>>>> } >>>>>> } >>>>> -- >>>>> --- >>>>> Jiaxun Yang >>>>> >>>> -- >>>> Sincerely yours, >>>> Mike.