From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 80173ECAAD3 for ; Thu, 1 Sep 2022 09:01:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1E09E6B0072; Thu, 1 Sep 2022 05:01:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 18FBC6B0073; Thu, 1 Sep 2022 05:01:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 031088D0001; Thu, 1 Sep 2022 05:01:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id E7E836B0072 for ; Thu, 1 Sep 2022 05:01:24 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id BC0FA160D3D for ; Thu, 1 Sep 2022 09:01:24 +0000 (UTC) X-FDA: 79862922888.16.0E53395 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by imf06.hostedemail.com (Postfix) with ESMTP id 2CC6F180046 for ; Thu, 1 Sep 2022 09:01:22 +0000 (UTC) Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 2818oSGd007656; Thu, 1 Sep 2022 09:01:09 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=message-id : date : subject : to : cc : references : from : in-reply-to : content-type : content-transfer-encoding : mime-version; s=pp1; bh=zuFrY9PbH6DI0qXYtE5kXVNqcrfdFquupbF8OFqj1Ds=; b=U0nxIRkgDSoWt3mmLfROeoeMWpTGS2sMEfQSe0wGZ/L5XCOZ8UcaviEOkX436pSCgOBf nK4yZKeBXhROqAcwvy93Q3Tyqt+1CgLoUkK1S9p+nW3Xy2Z6I/k97TuNMqTjVtNV7Zh9 OSpbVcI29rnDyne3IFthasqL+hUquHtwTrlplpceSO5r05w3+0opuIHTi4Hlj3+eXlmD Y7m7PfulfYfdrzMngzWi7MUMGYpxCAq9VzQL4emQTR/LXZR0VwUNoT4jJ876hlljJMcs +NNI5Sm6yuhMyMk/JkuWMECDuFbIap4ISe2S+HVltSNqqvh9QcvwPv3XjV7c/cn2ogpc gw== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3jask18a52-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 01 Sep 2022 09:01:09 +0000 Received: from m0098409.ppops.net (m0098409.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 2818pCso013578; Thu, 1 Sep 2022 09:01:08 GMT Received: from ppma03ams.nl.ibm.com (62.31.33a9.ip4.static.sl-reverse.com [169.51.49.98]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3jask189uu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 01 Sep 2022 09:01:08 +0000 Received: from pps.filterd (ppma03ams.nl.ibm.com [127.0.0.1]) by ppma03ams.nl.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 2818q0kH003829; Thu, 1 Sep 2022 09:00:52 GMT Received: from b06avi18626390.portsmouth.uk.ibm.com (b06avi18626390.portsmouth.uk.ibm.com [9.149.26.192]) by ppma03ams.nl.ibm.com with ESMTP id 3j7aw96fnc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 01 Sep 2022 09:00:52 +0000 Received: from d06av22.portsmouth.uk.ibm.com (d06av22.portsmouth.uk.ibm.com [9.149.105.58]) by b06avi18626390.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 2818vWGc38470042 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 1 Sep 2022 08:57:32 GMT Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 882DC4C059; Thu, 1 Sep 2022 09:00:49 +0000 (GMT) Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A5B514C04A; Thu, 1 Sep 2022 09:00:46 +0000 (GMT) Received: from [9.43.54.15] (unknown [9.43.54.15]) by d06av22.portsmouth.uk.ibm.com (Postfix) with ESMTP; Thu, 1 Sep 2022 09:00:46 +0000 (GMT) Message-ID: Date: Thu, 1 Sep 2022 14:30:45 +0530 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.2.0 Subject: Re: [PATCH] mm: hugetlb: eliminate memory-less nodes handling Content-Language: en-US To: Muchun Song , gregkh@linuxfoundation.org, rafael@kernel.org, mike.kravetz@oracle.com, akpm@linux-foundation.org, osalvador@suse.de, david@redhat.com, ying.huang@intel.com, rientjes@google.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org References: <20220901083023.42319-1-songmuchun@bytedance.com> From: Aneesh Kumar K V In-Reply-To: <20220901083023.42319-1-songmuchun@bytedance.com> Content-Type: text/plain; charset=UTF-8 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: KGevbamHqVRcBguNAm3gCn1t4HORL7b7 X-Proofpoint-GUID: _QRkGQR8gQA2pYu0ciVDtmKLtkdIG5Zv Content-Transfer-Encoding: 7bit X-Proofpoint-UnRewURL: 0 URL was un-rewritten MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-09-01_06,2022-08-31_03,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 suspectscore=0 spamscore=0 phishscore=0 bulkscore=0 mlxlogscore=999 mlxscore=0 impostorscore=0 lowpriorityscore=0 priorityscore=1501 malwarescore=0 clxscore=1011 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2207270000 definitions=main-2209010040 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1662022883; a=rsa-sha256; cv=none; b=l47vAeHl5NnTB0iH9xX2eaZRaxhKhU9i4w/Z24TuBiL2hBOdA8RmvXTFXU4EpPiXdJPKi9 TJ14eagxJyWnu5bCb+LmVqRhGez0Nko3AZoTFqZB2pciyGjuQ97zkLbAECmS2hOVNs2bFp dnlGEW4BzRcnfV5qxWQdAY1Sq+xT45w= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=U0nxIRkg; spf=pass (imf06.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com; dmarc=pass (policy=none) header.from=ibm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1662022883; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=zuFrY9PbH6DI0qXYtE5kXVNqcrfdFquupbF8OFqj1Ds=; b=qV6MPDL7td+mEybLgOw1qkPeYWnGvIgFELHwZCv7ws2Gc+4AgOywipWE77m6UKSxgc8sf2 PVUQiCQIc/ze9HH85EoN5YMCD13+L90uJ2VmoWO68vuSDwciKsT2c66XeRnzY9SC2NcBtC fcBzAi0rCjnmYk4jhle4eYFE/sPgbwo= X-Rspamd-Queue-Id: 2CC6F180046 Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=U0nxIRkg; spf=pass (imf06.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com; dmarc=pass (policy=none) header.from=ibm.com X-Rspam-User: X-Rspamd-Server: rspam01 X-Stat-Signature: sj5j8ttecbop7xk9kdkie97wppt1ai8i X-HE-Tag: 1662022882-939791 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 9/1/22 2:00 PM, Muchun Song wrote: > The memory-notify-based approach aims to handle meory-less nodes, however, it just adds > the complexity of code as pointed by David in thread [1]. The handling of memory-less > nodes is introduced by commit 4faf8d950ec4 ("hugetlb: handle memory hot-plug events"). > From its commit message, we cannot find any necessity of handling this case. So, we can > simply register/unregister sysfs entries in register_node/unregister_node to simlify the > code. Isn't that hotplug callback added because in hugetlb_register_all_nodes() we register sysfs nodes only for N_MEMORY nodes? > > https://lore.kernel.org/linux-mm/60933ffc-b850-976c-78a0-0ee6e0ea9ef0@redhat.com/ [1] > Suggested-by: David Hildenbrand > Signed-off-by: Muchun Song > --- > drivers/base/node.c | 7 +++++-- > include/linux/node.h | 5 +++++ > mm/hugetlb.c | 37 ++++++++++--------------------------- > 3 files changed, 20 insertions(+), 29 deletions(-) > > diff --git a/drivers/base/node.c b/drivers/base/node.c > index ed391cb09999..cf115d5a9b8a 100644 > --- a/drivers/base/node.c > +++ b/drivers/base/node.c > @@ -608,10 +608,12 @@ static int register_node(struct node *node, int num) > node->dev.groups = node_dev_groups; > error = device_register(&node->dev); > > - if (error) > + if (error) { > put_device(&node->dev); > - else > + } else { > + hugetlb_register_node(node); > compaction_register_node(node); > + } > I guess this will handle register of sysfs hugetlb files for new NUMA nodes after hugetlb_initialized = true; But what about N_CPU that can get memory added later. Do we need to update hugetlb_register_all_nodes() to handle N_ONLINE? > return error; > } > @@ -625,6 +627,7 @@ static int register_node(struct node *node, int num) > */ > void unregister_node(struct node *node) > { > + hugetlb_unregister_node(node); > compaction_unregister_node(node); > node_remove_accesses(node); > node_remove_caches(node); > diff --git a/include/linux/node.h b/include/linux/node.h > index 427a5975cf40..f5d41498c2bf 100644 > --- a/include/linux/node.h > +++ b/include/linux/node.h > @@ -138,6 +138,11 @@ extern void unregister_memory_block_under_nodes(struct memory_block *mem_blk); > extern int register_memory_node_under_compute_node(unsigned int mem_nid, > unsigned int cpu_nid, > unsigned access); > + > +#ifdef CONFIG_HUGETLBFS > +void hugetlb_register_node(struct node *node); > +void hugetlb_unregister_node(struct node *node); > +#endif > #else > static inline void node_dev_init(void) > { > diff --git a/mm/hugetlb.c b/mm/hugetlb.c > index d0617d64d718..722e862bb6be 100644 > --- a/mm/hugetlb.c > +++ b/mm/hugetlb.c > @@ -3898,6 +3898,7 @@ static void __init hugetlb_sysfs_init(void) > } > > #ifdef CONFIG_NUMA > +static bool hugetlb_initialized __ro_after_init; > > /* > * node_hstate/s - associate per node hstate attributes, via their kobjects, > @@ -3953,7 +3954,7 @@ static struct hstate *kobj_to_node_hstate(struct kobject *kobj, int *nidp) > * Unregister hstate attributes from a single node device. > * No-op if no hstate attributes attached. > */ > -static void hugetlb_unregister_node(struct node *node) > +void hugetlb_unregister_node(struct node *node) > { > struct hstate *h; > struct node_hstate *nhs = &node_hstates[node->dev.id]; > @@ -3983,19 +3984,22 @@ static void hugetlb_unregister_node(struct node *node) > * Register hstate attributes for a single node device. > * No-op if attributes already registered. > */ > -static int hugetlb_register_node(struct node *node) > +void hugetlb_register_node(struct node *node) > { > struct hstate *h; > struct node_hstate *nhs = &node_hstates[node->dev.id]; > int err; > > + if (!hugetlb_initialized) > + return; > + > if (nhs->hugepages_kobj) > - return 0; /* already allocated */ > + return; /* already allocated */ > > nhs->hugepages_kobj = kobject_create_and_add("hugepages", > &node->dev.kobj); > if (!nhs->hugepages_kobj) > - return -ENOMEM; > + return; > > for_each_hstate(h) { > err = hugetlb_sysfs_add_hstate(h, nhs->hugepages_kobj, > @@ -4005,28 +4009,9 @@ static int hugetlb_register_node(struct node *node) > pr_err("HugeTLB: Unable to add hstate %s for node %d\n", > h->name, node->dev.id); > hugetlb_unregister_node(node); > - return -ENOMEM; > + break; > } > } > - return 0; > -} > - > -static int __meminit hugetlb_memory_callback(struct notifier_block *self, > - unsigned long action, void *arg) > -{ > - int ret = 0; > - struct memory_notify *mnb = arg; > - int nid = mnb->status_change_nid; > - > - if (nid == NUMA_NO_NODE) > - return NOTIFY_DONE; > - > - if (action == MEM_GOING_ONLINE) > - ret = hugetlb_register_node(node_devices[nid]); > - else if (action == MEM_CANCEL_ONLINE || action == MEM_OFFLINE) > - hugetlb_unregister_node(node_devices[nid]); > - > - return notifier_from_errno(ret); > } > > /* > @@ -4038,11 +4023,9 @@ static void __init hugetlb_register_all_nodes(void) > { > int nid; > > - get_online_mems(); > - hotplug_memory_notifier(hugetlb_memory_callback, 0); > + hugetlb_initialized = true; > for_each_node_state(nid, N_MEMORY) Should this be for_each_online_node() ? > hugetlb_register_node(node_devices[nid]); > - put_online_mems(); > } > #else /* !CONFIG_NUMA */ >