From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25946C54E64 for ; Mon, 25 Mar 2024 22:53:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A9D2A6B0095; Mon, 25 Mar 2024 18:53:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A4CFE6B0096; Mon, 25 Mar 2024 18:53:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 914946B0098; Mon, 25 Mar 2024 18:53:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 81F656B0095 for ; Mon, 25 Mar 2024 18:53:14 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 58D26802BF for ; Mon, 25 Mar 2024 22:53:14 +0000 (UTC) X-FDA: 81937063908.23.FA2C626 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf11.hostedemail.com (Postfix) with ESMTP id 10D1940008 for ; Mon, 25 Mar 2024 22:53:11 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Keb4PkxS; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf11.hostedemail.com: domain of sj@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=sj@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711407192; a=rsa-sha256; cv=none; b=holpJAk64JVkoPlLHTA3wYUxEgw5IxRlAhkNKCsUeD3dE53rjcm9JifC2tfu2ZJ7NBsXg7 GLPX78clZ+zFwrlF2PN987GsLwRDgIC2GP/z5uxsukko3pUaRGckqWUB0Irpg4C2SL0pZu yXKFZydBsFD4ij03N8ArVQaNpDsmH3g= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Keb4PkxS; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf11.hostedemail.com: domain of sj@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=sj@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711407192; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=GGnWvhYNTW1ykZfR+jsYdwz0GxoBqDgiwJHZm65BPeU=; b=cDdtLOKhQbPLfxLW44noN/M3vECvbGEkA03rFjo5lGwA641qbR+VGmPQfqZaQCc89rs4Ta u8R5oAYpjI7A3mA10HbfsHtxaYeiQHfbf5QOe3yjgPN5tMdA1/fuVJVpjcswTkVfIiSV+B MF0r1nwyFdxHJ7A+7WHzT9ImscYOS2k= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id E8DC2CE1D9B; Mon, 25 Mar 2024 22:53:07 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 449C6C433F1; Mon, 25 Mar 2024 22:53:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1711407186; bh=ljVYVSahgnqQzJ4KwzCiQ0uzQsg+xQTx/hr9gGB+Iug=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Keb4PkxST0YJtRTWJGR3/6IYV4hiyPQbiDkAtjnGFDOcKDRmhP9T4PSsB5vYxOLJb f6nHJTY6WbxH6hUnbXFpH32SP8mvMoWBC18au9J7XPsD00v/QKiVeGSugyg9HkyMrx 6GciER951lyA1LFnabSPpK5pGzT7yL3u+NHjn5grhHxNKURmT+Z1q6PqAAVwdiUvmR yYibofWAuA5liLoUXKEOl0i0Ldkc3wBJ/qWqB634mWK834xMQbhj9L9j3NJh27vBaG gpHoPGF4E+KqfiLF7O1AlSyI2fDIISl2nT1fxgAQ7rCNxo63CZDyf8ROrQwixuoEjZ ppwiZ6N4NmMMg== From: SeongJae Park To: Honggyu Kim Cc: SeongJae Park , damon@lists.linux.dev, linux-mm@kvack.org, akpm@linux-foundation.org, apopple@nvidia.com, baolin.wang@linux.alibaba.com, dave.jiang@intel.com, hyeongtak.ji@sk.com, kernel_team@skhynix.com, linmiaohe@huawei.com, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, mathieu.desnoyers@efficios.com, mhiramat@kernel.org, rakie.kim@sk.com, rostedt@goodmis.org, surenb@google.com, yangx.jy@fujitsu.com, ying.huang@intel.com, ziy@nvidia.com, 42.hyeyoo@gmail.com Subject: Re: [RFC PATCH v2 0/7] DAMON based 2-tier memory management for CXL memory Date: Mon, 25 Mar 2024 15:53:03 -0700 Message-Id: <20240325225304.235736-1-sj@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240325120108.2328-1-honggyu.kim@sk.com> References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 10D1940008 X-Stat-Signature: 9369ukhx4o7be3jnajh4ddqp5yzagjq7 X-HE-Tag: 1711407191-763344 X-HE-Meta: U2FsdGVkX1+U/O6MZYhEK9KTRm/BTEMRf5BTBGZFAt2siCcRGL8Z/KhTv77SLJhKK1NqjT+Np5RR85sYh2h9tyb3Dgj8fWiIqVvsB+iEco2OfYMHshrOjr+QofOkKJEmTc0onSK17LhRMPS1ztsYSN2KFmxVtJ1FLYwuWkeQ3Am3l3ym/DJe2PYpwd4GF+JzILIhfLaEIfgm7qCNNQtvNjImVkC+w5nA4R85Rl6JpKWkrgHpwxaVo5XtJUfTc8bWBOb8tgl0ea7XtY7gv4j1gulltNkKyTO81fkVl6jXydv1lVc5OCXgdUUPyPom8rXrnofRQi0YhD0e9941XlupdkTBcoy92rZYANhU3CdCei7rzJyzB7wZCdImO3vOyq3JnkWWHGDOnuVt32w4ilnc+GuzJ5jBBIYgGE7rR/cgsCqMXl+s3ArsAaH+LnL4tprBBVbz996CJ+GgupXP1WijL7tlel1fzg36815EYPXonb3L6ri1fc6xTHDcZ0ZTFXEmVKx5ZEAyX4udBtjjUji53gLErIYn5uu5tVGVDMWBX5mRZ3SxmG8xA2w5MlkfK0A6pF95zYLykHtNOaYF/njQfgYx7o7ZeurjqfR9izlcwKCoHanFqi7x6pw8sxiuBepcdb9c6bDeXWVhW1hb1OfErG1/LG92sgjWZRenZu2Kn9GQyArjBDgnmn1L63vD9BOhu7bGiUFZDVTIwJPq8SWsGPzjtj2IjovrQQbZBI4YtoYj/lIWQevSjduS9w/9LmLjMqPoBYRJtmNDxGbDaiarqGm3NV34URoPjoIeUdoeoql1Eyd1PM5gd7JPYuzn+iHa1uctF0pKIx03+7PNBT07+FBd99R+xoIAG7VJBkgspf6fTHVrDnVvbZXB8JNPBYveDvJPlPV2raAT46e5hcxbV8vtE0dNvECNbhFd5xBEZK+ebbMIeXPyLrgyP6FJgMdjOrRgGXqxS49OY78sbE3 ZhQvXMTn QzcgatZq1ex5DLG4WzGzrST1xF9YA6uHYv9TT8siAnJ7CSAstFQBwvSK3j9EDWbEtYURZ+1vt5T9nTXYXD9YJbJQJvNlfPb1tWwpeNMM1qpJlIAiOh/KUZdLpdFZdyEKxE4clu5q/d2KasoK1wirTqt6g3dG1UhJ4tkzknira6KlzCSkQnDq3/PxqUEo2LmlrzdI6BTuTP8JOOsIWwkd4fkuc+yw48SZoj+2LKFEnjPZ6uuYkIwfRAmMh0MhE8WQd65coPP2E1XiihbWV55XA8q2EmH/PrxSnp7C+hdFfacn4ByTOB26WJbN+pbcJahBRh+233+cGgC/XPkqcIkCzcMixBz2qJIRD2GBZ23R7IiDs0o1tt4KlCAmHHpef0Jj8uatgrLFCIoa9sFk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, 25 Mar 2024 21:01:04 +0900 Honggyu Kim wrote: > Hi SeongJae, > > On Fri, 22 Mar 2024 09:32:23 -0700 SeongJae Park wrote: > > On Fri, 22 Mar 2024 18:02:23 +0900 Honggyu Kim wrote: [...] > > > > Honggyu joined DAMON Beer/Coffee/Tea Chat[1] yesterday, and we discussed about > > > > this patchset in high level. Sharing the summary here for open discussion. As > > > > also discussed on the first version of this patchset[2], we want to make single > > > > action for general page migration with minimum changes, but would like to keep > > > > page level access re-check. We also agreed the previously proposed DAMOS > > > > filter-based approach could make sense for the purpose. > > > > > > Thanks very much for the summary. I have been trying to merge promote > > > and demote actions into a single migrate action, but I found an issue > > > regarding damon_pa_scheme_score. It currently calls damon_cold_score() > > > for demote action and damon_hot_score() for promote action, but what > > > should we call when we use a single migrate action? > > > > Good point! This is what I didn't think about when suggesting that. Thank you > > for letting me know this gap! I think there could be two approach, off the top > > of my head. > > > > The first one would be extending the interface so that the user can select the > > score function. This would let flexible usage, but I'm bit concerned if this > > could make things unnecessarily complex, and would really useful in many > > general use case. > > I also think this looks complicated and may not be useful for general > users. > > > The second approach would be letting DAMON infer the intention. In this case, > > I think we could know the intention is the demotion if the scheme has a youg > > pages exclusion filter. Then, we could use the cold_score(). And vice versa. > > To cover a case that there is no filter at all, I think we could have one > > assumption. My humble intuition says the new action (migrate) may be used more > > for promotion use case. So, in damon_pa_scheme_score(), if the action of the > > given scheme is the new one (say, MIGRATE), the function will further check if > > the scheme has a filter for excluding young pages. If so, the function will > > use cold_score(). Otherwise, the function will use hot_score(). > > Thanks for suggesting many ideas but I'm afraid that I feel this doesn't > look good. Thinking it again, I think we can think about keep using > DAMOS_PROMOTE and DAMOS_DEMOTE, In other words, keep having dedicated DAMOS action for intuitive prioritization score function, or, coupling the prioritization with each action, right? I think this makes sense, and fit well with the documentation. The prioritization mechanism should be different for each action. For example, rarely accessed (colder) memory regions would be prioritized for page-out scheme action. In contrast, the colder regions would be deprioritized for huge page collapse scheme action. Hence, the prioritization mechanisms for each action are implemented in each DAMON operations set, together with the actions. In other words, each DAMOS action should allow users intuitively understand what types of regions will be prioritized. We already have such couples of DAMOS actions such as DAMOS_[NO]HUGEPAGE and DAMOS_LRU_[DE]PRIO. So adding a couple of action for this case sounds reasonable to me. And I think this is better and simpler than having the inferrence based behavior. That said, I concern if 'PROMOTE' and 'DEMOTE' still sound bit ambiguous to people who don't know 'demote_folio_list()' and its friends. Meanwhile, the name might sound too detail about what it does to people who know the functions, so make it bit unflexible. They might also get confused since we don't have 'promote_folio_list()'. To my humble understanding, what you really want to do is migrating pages to specific address range (or node) prioritizing the pages based on the hotness. What about, say, MIGRATE_{HOT,COLD}? > but I can make them directly call > damon_folio_young() for access check instead of using young filter. > > And we can internally handle the complicated combination such as demote > action sets "young" filter with "matching" true and promote action sets > "young" filter with "matching" false. IMHO, this will make the usage > simpler. I think whether to exclude young/non-young (maybe idle is better than non-young?) pages from the action is better to be decoupled for following reasons. Firstly, we want to check the page granularity youngness mainly because we found DAMON's monitoring result is not accurate enough for this use case. Or, we could say that's because you cannot wait until DAMON's monitoring result becomes accurate enough. For more detail, you could increase minimum age of your scheme's target access pattern. I show you set minimum age of your demote scheme as 30 seconds, but set promote scheme as 0 sec (https://github.com/skhynix/hmsdk/blob/main/tools/gen_config.py). Increasing the minimum age for promote scheme may reduce amount of wrong promotion. But I assume you don't want to do that, maybe because you want to make promotion fast. That's 100% valid use case in my opinion. And for such use case, making the DAMOS action to do the page granularity access check would be helpful. But if the user can set minimum age of two schemes large enough, or somehow DAMON's monitoring accuracy is much improved, this page granularity access check might not really required. Secondly, I think we might want to migrate pages to other nodes in coldest pages first way, but even if the page is young. One such case would be when the top tier cannot accommodate all young pages. Especially when there are no small number of tiers, I think this could happen. That is, we would prefer migrating coldest page first, but that depend on only relative temperature and hence young pages could also need to be migrated. Of course, this would be the real case only if the user is convinced with DAMON's monitoring accuracy. > > I would like to hear how you think about this. So, to summarize my humble opinion, 1. I like the idea of having two actions. But I'd like to use names other than 'promote' and 'demote'. 2. I still prefer having a filter for the page granularity access re-check. > > > So I'd more prefer the second approach. I think it would be not too late to > > consider the first approach after waiting for it turns out more actions have > > such ambiguity and need more general interface for explicitly set the score > > function. > > I will join the DAMON Beer/Coffee/Tea Chat tomorrow as scheduled so I > can talk more about this issue. Looking forward to chatting with you :) Thanks, SJ > > Thanks, > Honggyu > > > > > Thanks, > > SJ > > > > [...]