From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7029BC43334 for ; Wed, 8 Jun 2022 08:29:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 10D018D001B; Wed, 8 Jun 2022 04:29:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0BCC98D0009; Wed, 8 Jun 2022 04:29:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EEDBD8D001B; Wed, 8 Jun 2022 04:29:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id DFFCF8D0009 for ; Wed, 8 Jun 2022 04:29:34 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay13.hostedemail.com (Postfix) with ESMTP id BA31460DF3 for ; Wed, 8 Jun 2022 08:29:34 +0000 (UTC) X-FDA: 79554394668.08.8199B84 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by imf23.hostedemail.com (Postfix) with ESMTP id CD24D140052 for ; Wed, 8 Jun 2022 08:29:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1654676973; x=1686212973; h=message-id:subject:from:to:cc:date:in-reply-to: references:mime-version:content-transfer-encoding; bh=6M2Lzi3AGThX+50BydU84KfhNGFSIyUBnMAloNA6clY=; b=DTl8z7inzRnpjjCbi9zJx5DjHo46gqsptvHZfTRndX/h7+zVgsBzjFG4 sIlELX36pEpgFzVk9ppF4eLOuhvO7xvmYqvB4G5JmyaFZtqWl11SduyHW jemXmAsUiQCqc9w/TjUN8BsYKp+i9vpbw7Go+G+2YIILNxC2gr2XL1BHz ifsX2fCTphhowceDXfdDSi59KVMau+cWeMWPh/I1xMN0KxxuBf9cuw61h XtW2uItMyylEurDXYKOz1KEHL7EaJ6ZSAuxLKNUmV+yVzvh01zbGbUfmh /8jkN8xC7S9+KdOWcJoPWLt0R3BZytAwDAYiaZAcuT8K4K/XMDBRkE2V/ w==; X-IronPort-AV: E=McAfee;i="6400,9594,10371"; a="256639542" X-IronPort-AV: E=Sophos;i="5.91,285,1647327600"; d="scan'208";a="256639542" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Jun 2022 01:29:32 -0700 X-IronPort-AV: E=Sophos;i="5.91,285,1647327600"; d="scan'208";a="579961629" Received: from xding11-mobl.ccr.corp.intel.com ([10.254.214.239]) by orsmga007-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Jun 2022 01:29:26 -0700 Message-ID: <6be904db28e3be0b0455ae750b595ee76edd08cf.camel@intel.com> Subject: Re: [RFC PATCH v4 2/7] mm/demotion: Expose per node memory tier to sysfs From: Ying Huang To: Aneesh Kumar K V , linux-mm@kvack.org, akpm@linux-foundation.org Cc: Greg Thelen , Yang Shi , Davidlohr Bueso , Tim C Chen , Brice Goglin , Michal Hocko , Linux Kernel Mailing List , Hesham Almatary , Dave Hansen , Jonathan Cameron , Alistair Popple , Dan Williams , Feng Tang , Jagdish Gediya , Baolin Wang , David Rientjes Date: Wed, 08 Jun 2022 16:29:23 +0800 In-Reply-To: References: <20220527122528.129445-1-aneesh.kumar@linux.ibm.com> <20220527122528.129445-3-aneesh.kumar@linux.ibm.com> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.38.3-1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Stat-Signature: 6c1ehhwst6qiw1gorhketzft51tiqs3q X-Rspam-User: Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=DTl8z7in; dmarc=pass (policy=none) header.from=intel.com; spf=none (imf23.hostedemail.com: domain of ying.huang@intel.com has no SPF policy when checking 192.55.52.136) smtp.mailfrom=ying.huang@intel.com X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: CD24D140052 X-HE-Tag: 1654676973-416917 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, 2022-06-08 at 13:55 +0530, Aneesh Kumar K V wrote: > On 6/8/22 12:48 PM, Ying Huang wrote: > > On Fri, 2022-05-27 at 17:55 +0530, Aneesh Kumar K.V wrote: > > > From: Jagdish Gediya > > > > > > Add support to read/write the memory tierindex for a NUMA node. > > > > > > /sys/devices/system/node/nodeN/memtier > > > > > > where N = node id > > > > > > When read, It list the memory tier that the node belongs to. > > > > > > When written, the kernel moves the node into the specified > > > memory tier, the tier assignment of all other nodes are not > > > affected. > > > > > > If the memory tier does not exist, writing to the above file > > > create the tier and assign the NUMA node to that tier. > > > > > > mutex memory_tier_lock is introduced to protect memory tier > > > related chanegs as it can happen from sysfs as well on hot > > > plug events. > > > > > > Signed-off-by: Jagdish Gediya > > > Signed-off-by: Aneesh Kumar K.V > > > --- > > >   drivers/base/node.c | 35 ++++++++++++++ > > >   include/linux/migrate.h | 4 +- > > >   mm/migrate.c | 103 ++++++++++++++++++++++++++++++++++++++++ > > >   3 files changed, 141 insertions(+), 1 deletion(-) > > > > > > diff --git a/drivers/base/node.c b/drivers/base/node.c > > > index ec8bb24a5a22..cf4a58446d8c 100644 > > > --- a/drivers/base/node.c > > > +++ b/drivers/base/node.c > > > @@ -20,6 +20,7 @@ > > >   #include > > >   #include > > >   #include > > > +#include > > >    > > > > > > > > > > > > > > >   static struct bus_type node_subsys = { > > >    .name = "node", > > > @@ -560,11 +561,45 @@ static ssize_t node_read_distance(struct device *dev, > > >   } > > >   static DEVICE_ATTR(distance, 0444, node_read_distance, NULL); > > >    > > > > > > > > > > > > > > > +#ifdef CONFIG_TIERED_MEMORY > > > +static ssize_t memtier_show(struct device *dev, > > > + struct device_attribute *attr, > > > + char *buf) > > > +{ > > > + int node = dev->id; > > > + > > > + return sysfs_emit(buf, "%d\n", node_get_memory_tier(node)); > > > +} > > > + > > > +static ssize_t memtier_store(struct device *dev, > > > + struct device_attribute *attr, > > > + const char *buf, size_t count) > > > +{ > > > + unsigned long tier; > > > + int node = dev->id; > > > + > > > + int ret = kstrtoul(buf, 10, &tier); > > > + if (ret) > > > + return ret; > > > + > > > + ret = node_reset_memory_tier(node, tier); > > > + if (ret) > > > + return ret; > > > + > > > + return count; > > > +} > > > + > > > +static DEVICE_ATTR_RW(memtier); > > > +#endif > > > + > > >   static struct attribute *node_dev_attrs[] = { > > >    &dev_attr_meminfo.attr, > > >    &dev_attr_numastat.attr, > > >    &dev_attr_distance.attr, > > >    &dev_attr_vmstat.attr, > > > +#ifdef CONFIG_TIERED_MEMORY > > > + &dev_attr_memtier.attr, > > > +#endif > > >    NULL > > >   }; > > >    > > > > > > > > > > > > > > > diff --git a/include/linux/migrate.h b/include/linux/migrate.h > > > index 0ec653623565..d37d1d5dee82 100644 > > > --- a/include/linux/migrate.h > > > +++ b/include/linux/migrate.h > > > @@ -177,13 +177,15 @@ enum memory_tier_type { > > >   }; > > >    > > > > > > > > > > > > > > >   int next_demotion_node(int node); > > > - > > >   extern void migrate_on_reclaim_init(void); > > >   #ifdef CONFIG_HOTPLUG_CPU > > >   extern void set_migration_target_nodes(void); > > >   #else > > >   static inline void set_migration_target_nodes(void) {} > > >   #endif > > > +int node_get_memory_tier(int node); > > > +int node_set_memory_tier(int node, int tier); > > > +int node_reset_memory_tier(int node, int tier); > > >   #else > > >   #define numa_demotion_enabled false > > >   static inline int next_demotion_node(int node) > > > diff --git a/mm/migrate.c b/mm/migrate.c > > > index f28ee93fb017..304559ba3372 100644 > > > --- a/mm/migrate.c > > > +++ b/mm/migrate.c > > > @@ -2132,6 +2132,7 @@ static struct bus_type memory_tier_subsys = { > > >    .dev_name = "memtier", > > >   }; > > >    > > > > > > > > > > > > > > > +DEFINE_MUTEX(memory_tier_lock); > > >   static struct memory_tier *memory_tiers[MAX_MEMORY_TIERS]; > > >    > > > > > > > > > > > > > > >   static ssize_t nodelist_show(struct device *dev, > > > @@ -2225,6 +2226,108 @@ static const struct attribute_group *memory_tier_attr_groups[] = { > > >    NULL, > > >   }; > > >    > > > > > > > > > > > > > > > +static int __node_get_memory_tier(int node) > > > +{ > > > + int tier; > > > + > > > + for (tier = 0; tier < MAX_MEMORY_TIERS; tier++) { > > > + if (memory_tiers[tier] && node_isset(node, memory_tiers[tier]->nodelist)) > > > + return tier; > > > + } > > > + > > > + return -1; > > > +} > > > + > > > +int node_get_memory_tier(int node) > > > +{ > > > + int tier; > > > + > > > + /* > > > + * Make sure memory tier is not unregistered > > > + * while it is being read. > > > + */ > > > + mutex_lock(&memory_tier_lock); > > > + > > > + tier = __node_get_memory_tier(node); > > > + > > > + mutex_unlock(&memory_tier_lock); > > > + > > > + return tier; > > > +} > > > + > > > +int __node_set_memory_tier(int node, int tier) > > > +{ > > > + int ret = 0; > > > + /* > > > + * As register_memory_tier() for new tier can fail, > > > + * try it before modifying existing tier. register > > > + * tier makes tier visible in sysfs. > > > + */ > > > + if (!memory_tiers[tier]) { > > > + ret = register_memory_tier(tier); > > > + if (ret) { > > > + goto out; > > > + } > > > + } > > > + > > > + node_set(node, memory_tiers[tier]->nodelist); > > > + > > > +out: > > > + return ret; > > > +} > > > + > > > +int node_reset_memory_tier(int node, int tier) > > > > I think "reset" isn't a good name here. Maybe something like "change" > > or "move"? > > > > how about node_update_memory_tier()? That sounds OK for me. Best Regards, Huang, Ying