From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.3 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6755AC433B4 for ; Wed, 14 Apr 2021 08:09:00 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E2467611F0 for ; Wed, 14 Apr 2021 08:08:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E2467611F0 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6B8056B0071; Wed, 14 Apr 2021 04:08:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 668C66B0073; Wed, 14 Apr 2021 04:08:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5098D6B0074; Wed, 14 Apr 2021 04:08:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0096.hostedemail.com [216.40.44.96]) by kanga.kvack.org (Postfix) with ESMTP id 34A1B6B0071 for ; Wed, 14 Apr 2021 04:08:59 -0400 (EDT) Received: from smtpin32.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id E335F4E1F7B4 for ; Wed, 14 Apr 2021 08:08:58 +0000 (UTC) X-FDA: 78030246756.32.5A8AAB2 Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by imf15.hostedemail.com (Postfix) with ESMTP id BB2DCA0003A3 for ; Wed, 14 Apr 2021 08:08:56 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 2CAD4AEC2; Wed, 14 Apr 2021 08:08:57 +0000 (UTC) Date: Wed, 14 Apr 2021 10:08:54 +0200 From: Oscar Salvador To: Wei Xu Cc: Dave Hansen , Linux MM , Linux Kernel Mailing List , Yang Shi , David Rientjes , Huang Ying , Dan Williams , David Hildenbrand Subject: Re: [PATCH 02/10] mm/numa: automatically generate node migration order Message-ID: <20210414080849.GA20886@linux> References: <20210401183216.443C4443@viggo.jf.intel.com> <20210401183219.DC1928FA@viggo.jf.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) X-Rspamd-Queue-Id: BB2DCA0003A3 X-Stat-Signature: qmkck5cnkojhg519ongtkaju9f6wczsr X-Rspamd-Server: rspam02 Received-SPF: none (suse.de>: No applicable sender policy available) receiver=imf15; identity=mailfrom; envelope-from=""; helo=mx2.suse.de; client-ip=195.135.220.15 X-HE-DKIM-Result: none/none X-HE-Tag: 1618387736-643746 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Apr 09, 2021 at 08:07:08PM -0700, Wei Xu wrote: > On Thu, Apr 1, 2021 at 11:35 AM Dave Hansen wrote: > > + * When Node 0 fills up, its memory should be migrated to > > + * Node 1. When Node 1 fills up, it should be migrated to > > + * Node 2. The migration path start on the nodes with the > > + * processors (since allocations default to this node) and > > + * fast memory, progress through medium and end with the > > + * slow memory: > > + * > > + * 0 -> 1 -> 2 -> stop > > + * 3 -> 4 -> 5 -> stop > > + * > > + * This is represented in the node_demotion[] like this: > > + * > > + * { 1, // Node 0 migrates to 1 > > + * 2, // Node 1 migrates to 2 > > + * -1, // Node 2 does not migrate > > + * 4, // Node 3 migrates to 4 > > + * 5, // Node 4 migrates to 5 > > + * -1} // Node 5 does not migrate > > + */ > > In this example, if we want to support multiple nodes as the demotion > target of a source node, we can group these nodes into three tiers > (classes): > > fast class: > 0 -> {1, 4} // 1 is the preferred > 3 -> {4, 1} // 4 is the preferred > > medium class: > 1 -> {2, 5} // 2 is the preferred > 4 -> {5, 2} // 5 is the preferred > > slow class: > 2 -> stop > 5 -> stop Hi Wei Xu, I have some questions about it Fast class/memory are pictured as those nodes with CPUs, while Slow class/memory are PMEM, right? Then, what stands for medium class/memory? In Dave's example, list is created in a way that stays local to the socket, and we go from the fast one to the slow one. In yours, lists are created taking the fastest nodes from all sockets and we work our way down, which means have cross-socket nodes in the list. How much of a penalty is that? And while I get your point, I am not sure if that is what we pretend here. This patchset aims to place cold pages that are about to be reclaim in slower nodes to give them a second chance, while your design seems more to have kind of different memory clases and be able to place applications in one of those tiers depending on its demands or sysadmin-demand. Could you expand some more? -- Oscar Salvador SUSE L3