From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A50E8C05027 for ; Sun, 12 Feb 2023 11:03:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CB2076B0073; Sun, 12 Feb 2023 06:03:25 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C62A26B0074; Sun, 12 Feb 2023 06:03:25 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B02AC6B0075; Sun, 12 Feb 2023 06:03:25 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 9EBD76B0073 for ; Sun, 12 Feb 2023 06:03:25 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 44C221A0A3C for ; Sun, 12 Feb 2023 11:03:25 +0000 (UTC) X-FDA: 80458353570.23.1E06B2F Received: from mail-pf1-f175.google.com (mail-pf1-f175.google.com [209.85.210.175]) by imf22.hostedemail.com (Postfix) with ESMTP id 7D23CC000B for ; Sun, 12 Feb 2023 11:03:21 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=YrKuy2r3; spf=pass (imf22.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.210.175 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=none) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676199803; a=rsa-sha256; cv=none; b=CghBczTWIORGtXH/pv4ie7JVrkQfxNpjFUp+abEUNo7NGlcHIb/19TgnjBbmvUi769m1Tp JRZ6VW+GGy/qCtUjP1eghmgiOloWLb3MNanNBfl2RWqfKRGP23upiRzFxNQP1MMnjeF0vW ge2iGk3huLdjWFucwOGbWKpoebOwqds= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=YrKuy2r3; spf=pass (imf22.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.210.175 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=none) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676199803; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=tcfVzYwQPANvaQ3YEeKrIrZyoY6D1sHP+o29Dg4fwsM=; b=yu1WTF5NYEtyec5u03z6SZlfmzytq6duJ/9k7ajtajGwXPlOr+bjEiw2alM2XGthCfuDeS yrAdnLte4BTdNd4nefFbXQMpUxcUY75FL5F5GbPbLqI96tee0eIs4Nf+FW2gf6q+ULr8AG dCCe1EdaLKrbpUF+aLWbsGn+yYsZsBE= Received: by mail-pf1-f175.google.com with SMTP id z3so30290pfw.7 for ; Sun, 12 Feb 2023 03:03:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=tcfVzYwQPANvaQ3YEeKrIrZyoY6D1sHP+o29Dg4fwsM=; b=YrKuy2r3nr3FtUgXY2Ju0cIDDVMp7LcjKJIG0PEay5WIdY69Xc+RhLvWAH/9bvDSbu p5pESBkU0NHzCSGY1ez0ObLpmjEo/kpbHD41PQnPDPgFR9/kedCc5Mas0JH45EaRxSmN v9Zbe7qfHf6JmQadL2st0BI7ovpWUscNDZSJZm9H/RbGwr3IX0p4ELNbdo6h62VVmbLx NhmH+OMZn9MK1peCtjm5dwGiOll//I3XXV81pez+8FI2Y4wOkvTmgfiLtcdGuKKsEaeP Mc0EIUz1Qj6amk6iv63ykk6EX3uW2SbttffHgKEOre0LMHJEYxqzDgfx3MXwOvlpX2rK D4/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=tcfVzYwQPANvaQ3YEeKrIrZyoY6D1sHP+o29Dg4fwsM=; b=iuhLucIQKviN2n8MwNk8y1dfuRpepw3K5ntJH546eYT76oekFZgkBfpMGhe4nsxV0L dKV5vQsQl/poqORIxvpescsV66axWNW/Kv+87m4PcjIx3uodZiY0kDLHW0s+YNOEgnQE fEhOldJf29oJSaSyMz0mKd80CRsp/2PyEcVSD/sSI85/m9iY0kUrDU33w8pUiPFVqEc1 fgC+UMfRv8AMXZC59XIM/C783xRbb5g3vKNT30JSGa19cF9c5fMH2eguuW5GJ6y25I5/ 0BdF4zH0oSV5aT8qOWO4ImwO6wc16EFZdFq4TmRvO/9ivPQ4rEFRwgVvvs2kbuYUkYZS 93nw== X-Gm-Message-State: AO0yUKVyBScbgwDb0tbILute6ZacbcVI5zKzsoPNC6f3C1aayUrj5uAC Mw+qsJs9UR3tRULIZOfH0dlIJw== X-Google-Smtp-Source: AK7set9RfIiai5AI9qdkqcCzbp6uV27HRQuTSUUWmvUp3PckXHJ0mew1kpJIQFdznV9FO7tj6tWrxA== X-Received: by 2002:a62:86c2:0:b0:5a8:abe2:fee2 with SMTP id x185-20020a6286c2000000b005a8abe2fee2mr1342985pfd.2.1676199800095; Sun, 12 Feb 2023 03:03:20 -0800 (PST) Received: from localhost.localdomain ([139.177.225.228]) by smtp.gmail.com with ESMTPSA id z20-20020aa791d4000000b0058e08796e98sm5963987pfa.196.2023.02.12.03.03.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 12 Feb 2023 03:03:19 -0800 (PST) From: Qi Zheng To: akpm@linux-foundation.org, vbabka@suse.cz Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Qi Zheng , Teng Hu Subject: [PATCH] mm: page_alloc: don't allocate page from memoryless nodes Date: Sun, 12 Feb 2023 19:03:05 +0800 Message-Id: <20230212110305.93670-1-zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspam-User: X-Rspamd-Queue-Id: 7D23CC000B X-Rspamd-Server: rspam01 X-Stat-Signature: hobpee65xeq3399owqdwnosp6pzubkjf X-HE-Tag: 1676199801-345485 X-HE-Meta: U2FsdGVkX19lcJ1NWZ5gcVg1zaWFMRGHUJD5QRTnidP5kqNUbFnlw+wxQqklFahD7MEzbw2WxTU2eu/AVxhSgtPRuUIElAXnxrxT7FNiOy8j+POc0WZ40bJDiFsJrpCuh7oiG2mj8FHa5ncJSgCJTzq3VpKrdT7KgEr3B6PYGD6mSCaCxAlg/lmYifk9/qskWQib0K8ppGCFpBXTrhIKfmUrKeUV7Na6Q4sl++ns3l5TfIBbX6lNCdL/6FS3y8JLD2oG3JChsJarIngR3mPF00dCL+Iwa9mDNfekaudZ+COgTzN4XP7CG5uxxGQowI+m0AVAiLyFrB2kStuI8sOmaV3A7r/nlBNXk+idEd415aBA5zZWgmmxrfxIbRGR93KrmBkg66lzjbndXg8L//5zYpFPdO2EZCVMEDP4vJe1b69DgvTstqCeLq45Z7iJ+H13FgWWI26sf2OVt8myE28AGRZA1i4OtID4razqTD2cSkcUjj/Dz1laisHtHd7Qnj54pgsPrwxxRMmrIqrAMTX6cAQf8HACEfWJtFWY8xpVelXYo4IkfBwwT4SqjvHRWYUb4VzfL05CLcfvz36UcBoQmkGrtWQb8ltk8VUHLqjQvMSCyFs/HQoz3KXGBUm8lRcjkn9JOl7HkpI4iCBxfSM7Yxg7FvdktrUPHhU/uuEReJLm0xPvYUSoiK3wshzvWIBzBDQvMZ1QeFgw/o4QL/BKuQnZ8tKdr+pQapS7IsPxl4CDEdhloJYwXBsOwba54SCm285w0cH5nRj2ffYgBWUamkRU8mxwFgzJgbXRCYB/YdyxaehBDggFcd1oqJBoRMKnulh6G3kMg6+pXqQRUlL3sHJg+5U7jyiDQOuWEQ1Hq2M/M0TlOL+awpZ0nERwh/xzFMkqtAeTSTCo2K8vVzYU5Ip9VdQZ29Uxmib18KkQrp0IUS8dpuL9wCxsz1fo5In/cS/v4Xk8359d96/fTz7 WWKX2eIP yaj7O2Cxg4i2BPV7NY6Uk93vuyRzT1GiR+pcNa3pHNJGSZiKwFFCNc0x5gHSoTZiC1Rm3F0NV6mF+GMZ+7SeJCqzi1hhCU1fH6R8HowewBtOOUrqfTARRQeNtBF5+fJnGqlN9fJAJ20Ckue9kXYImJMCyy4K5ylUNc3LaRJq+VJiuKtT19EH8cCq8i9VyCz/RYQpe/Kqg7dnW3S/5RL0bjuWphs3KgiJ+dZn/Yqn29XY6xIpxJG9ChM2U9dJhKFP9eBK0v6mISfd9K5tUQWC0+oN5je6E8BUEZ/ZfEnU89nNYGVR5OgwSOVh30OgxEOibKk4Z5FdeHlRsStm3JJnpSrGQvnlu428isjU3Rq76NjsH0l0NB9kaOD+2L9PkbIG1wm4ioAIxnF8jpKKUbrPv6jzp/B5iQ3e3JxQCa2yTbmcmsd09d2Pcpc50geG//wyAuOgKTaBg10ucIpyyFktflylmtHt2qhrNZLOjkmh8lOwLzFVJyQP89l/z+Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In x86, numa_register_memblks() is only interested in those nodes which have enough memory, so it skips over all nodes with memory below NODE_MIN_SIZE (treated as a memoryless node). Later on, we will initialize these memoryless nodes (allocate pgdat in free_area_init() and build zonelist etc), and will online these nodes in init_cpu_to_node() and init_gi_nodes(). After boot, these memoryless nodes are in N_ONLINE state but not in N_MEMORY state. But we can still allocate pages from these memoryless nodes. In SLUB, we only process nodes in the N_MEMORY state, such as allocating their struct kmem_cache_node. So if we allocate a page from the memoryless node above to SLUB, the struct kmem_cache_node of the node corresponding to this page is NULL, which will cause panic. For example, if we use qemu to start a two numa node kernel, one of the nodes has 2M memory (less than NODE_MIN_SIZE), and the other node has 2G, then we will encounter the following panic: [ 0.149844] BUG: kernel NULL pointer dereference, address: 0000000000000000 [ 0.150783] #PF: supervisor write access in kernel mode [ 0.151488] #PF: error_code(0x0002) - not-present page <...> [ 0.156056] RIP: 0010:_raw_spin_lock_irqsave+0x22/0x40 <...> [ 0.169781] Call Trace: [ 0.170159] [ 0.170448] deactivate_slab+0x187/0x3c0 [ 0.171031] ? bootstrap+0x1b/0x10e [ 0.171559] ? preempt_count_sub+0x9/0xa0 [ 0.172145] ? kmem_cache_alloc+0x12c/0x440 [ 0.172735] ? bootstrap+0x1b/0x10e [ 0.173236] bootstrap+0x6b/0x10e [ 0.173720] kmem_cache_init+0x10a/0x188 [ 0.174240] start_kernel+0x415/0x6ac [ 0.174738] secondary_startup_64_no_verify+0xe0/0xeb [ 0.175417] [ 0.175713] Modules linked in: [ 0.176117] CR2: 0000000000000000 In addition, we can also encountered this panic in the actual production environment. We set up a 2c2g container with two numa nodes, and then reserved 128M for kdump, and then we can encountered the above panic in the kdump kernel. To fix it, we can filter memoryless nodes when allocating pages. Signed-off-by: Qi Zheng Reported-by: Teng Hu --- mm/page_alloc.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 588555754601..b9cce56f4e21 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -4188,6 +4188,11 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int order, int alloc_flags, (alloc_flags & ALLOC_CPUSET) && !__cpuset_zone_allowed(zone, gfp_mask)) continue; + + /* Don't allocate page from memoryless nodes. */ + if (!node_state((zone_to_nid(zone)), N_MEMORY)) + continue; + /* * When allocating a page cache page for writing, we * want to get it from a node that is within its dirty -- 2.20.1