From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-vk1-f196.google.com (mail-vk1-f196.google.com [209.85.221.196]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 803CA36923B for ; Mon, 23 Feb 2026 12:38:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.196 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1771850296; cv=none; b=KnsKk4PyAo++oNsaC8gzB7OYopH0mYG9S1G1lchCh2+oxWcWzQIBb4jd4VSMkR+tN2E33Tdplo02IKu1VUZOHD2aTLFxsXrQcs/3qX9Ji5tWuoRxHkqDTySIuzg3hVgU7R9q0nCWVPC2osw8OOr336vC3Tr4wRiKYsr3gYcgcyE= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1771850296; c=relaxed/simple; bh=9A3KDPULYLSQF2TtBzqTrGQAnOGp35GVxCILsbUiNKE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Sklc3KBbcYVVSS7Rx9Ub12ztTbLOi7bFqumWheJLHXZvIFRVEnrcK1ih4kD2wDW/2WgZxcnCwTuGXJrjTQrtG+O7X+KfCasPDtQzzOyedSpkPdPnasf1Y2iAwfBpN8GUml6uLJGHVbqAa4zz1jtMeou/Dz5W2IjsR/HZZtPiRCk= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=R8ZmwT+2; arc=none smtp.client-ip=209.85.221.196 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="R8ZmwT+2" Received: by mail-vk1-f196.google.com with SMTP id 71dfb90a1353d-5670ea44187so3361512e0c.0 for ; Mon, 23 Feb 2026 04:38:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1771850293; x=1772455093; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=GYNB6/pZb33YeAKubCBWRXUx4k4dpplKbNpn3dWK4iI=; b=R8ZmwT+2qa57eLHfePgqwvbuSNXeS2HYiY4wzME2y1M+FqbvLaOhSHrwY+gYTOx96b 9FafENdul7AzTjJP2lOOVEpwEBWBxy+tuPorYpO7NUgX8Jh42NYtVq7dhEKQmZWBS76m rAfKi0nbfzKJydlFKVite1b2RJtWY9zjCGM9/mwo8E+mo6Fw5i7UV5lSusyBO99zvrH/ DCAhvbQW76+Z7pB/VtY2+iMrfM7CFueipPwLmNiN8Y4ZfI1RwOFYQa1TLiMutJ/JyX1l kpWP3MEbDNDjyEDMj+k/Y9QWE6PdewmSPpHS+Cx98E90LIQfuI5cj2sPX6Czn7o/nh6l wlFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1771850293; x=1772455093; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=GYNB6/pZb33YeAKubCBWRXUx4k4dpplKbNpn3dWK4iI=; b=pJINX+rp4Xhv4PZ+phY/+hBWYyCj5ZMZxAn1vbO1niq17TZb1vOrbqxQ+8Gi1KvL+4 Iijf7P4kVPlkJ0yd7+MyQkxgUZMcWOesAoSfQQy7m0Zods1KFA14sBXTvhAlcWR+Hl2l nMydvl0I5rial5/ivil0nVBtTzjbh+t3hO41pXxZO4ocRAb9YydPqtQnQxVEtZXEIg93 b8C3k813Ylc17n10JrAl4XSUrWg93X8FVXLmnyp6ksXhxxhgysyscPUwrmuBnIuCSDwW VYhWowDxgE5sQs/pZxIURWwnyKliGZt3Z5Bo/VivI5cE2iT9fLbifO62bkx5e6uVcJ0t Lj/Q== X-Forwarded-Encrypted: i=1; AJvYcCVfiTzKLoM9J0D1tK3v0b+e/liult5J4hQXzFhARdradh03qfB9hk/IVsMtJfxqPYGG3hdGM3bGnQ4YSRY=@vger.kernel.org X-Gm-Message-State: AOJu0YytucLH2DHPxIqSo8njN74B3zu6N157lDzJ0635FI5aIyZDHT1K D1oBwwjrLHIRHvUSruEV5cEjjvUZlWcarO/XY+OyKFez1FALVcvUHhwIxYl3pXkzqQ== X-Gm-Gg: AZuq6aLTuDshjf/sB2rkMeDkk8ng93p30J8qGgTFq7I6zfQ1BBlk6kaauI9fcQqoddo WymNJxLpHzQ0TcPSjQ1WKKjlr4JPWNHkAZ/C3PlktCX1YqF39eq4gUlLkPjYYTV5cEcZA6TR7Ke OmdqSgky6Uf/bKZTHftXdOsRFsS9UNlLUv1EZcBv9v2Fju0Li6H9Vto2UE3QInEj+zb317c+AnT ycin5RttSfzSawjIf0MrakuSBCFrtKphl6uDvYZ0K4ahinpuLTqXKN2kBN09stAOTLzT2FYg337 NAQWBs6RB1w4/G1x4B7Dt7xZZYwCngwFK/YsuHc+gh3Wif9Y6SaGt472NcY3sKevWWvwBlChHTn 2zwHY/OUC1vGOHWaZI+VQ1n28U7CellviAak1AQHEMYjyvut2Heli/uE0xyl2oiKb/fGNZU9+nA X9+YkgGozLWhTKhOnvbLU9WrUUudQUDHZv6Wvo56vqHJNP X-Received: by 2002:a05:6a20:cc97:b0:35e:11ff:45c1 with SMTP id adf61e73a8af0-39545ebee1cmr6462789637.18.1771849983299; Mon, 23 Feb 2026 04:33:03 -0800 (PST) Received: from LAPTOP-FDBL0TVI.localdomain ([49.37.157.71]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-c70b71a73e1sm7454739a12.13.2026.02.23.04.32.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Feb 2026 04:33:02 -0800 (PST) From: Ravi Jonnalagadda To: sj@kernel.org, damon@lists.linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org Cc: akpm@linux-foundation.org, corbet@lwn.net, bijan311@gmail.com, ajayjoshi@micron.com, honggyu.kim@sk.com, yunjeong.mun@sk.com, Ravi Jonnalagadda Subject: [RFC PATCH v3 3/4] mm/damon: add node_eligible_mem_bp and node_ineligible_mem_bp goal metrics Date: Mon, 23 Feb 2026 12:32:31 +0000 Message-ID: <20260223123232.12851-4-ravis.opensrc@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260223123232.12851-1-ravis.opensrc@gmail.com> References: <20260223123232.12851-1-ravis.opensrc@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add new quota goal metrics for memory tiering that track scheme-eligible (hot) memory distribution across NUMA nodes: - DAMOS_QUOTA_NODE_ELIGIBLE_MEM_BP: ratio of hot memory on a node - DAMOS_QUOTA_NODE_INELIGIBLE_MEM_BP: ratio of hot memory NOT on a node These complementary metrics enable push-pull migration schemes that maintain a target hot memory distribution. For example, to keep 30% of hot memory on CXL node 1: - PUSH scheme (DRAM→CXL): node_eligible_mem_bp, nid=1, target=3000 Activates when node 1 has less than 30% hot memory - PULL scheme (CXL→DRAM): node_ineligible_mem_bp, nid=1, target=7000 Activates when node 1 has more than 30% hot memory Together with the TEMPORAL goal tuner, the schemes converge to equilibrium at the target distribution. The metrics use detected eligible bytes per node, calculated by summing the size of regions that match the scheme's access pattern (size, nr_accesses, age) on each NUMA node. Suggested-by: SeongJae Park Signed-off-by: Ravi Jonnalagadda --- include/linux/damon.h | 6 ++ mm/damon/core.c | 123 ++++++++++++++++++++++++++++++++++++++- mm/damon/sysfs-schemes.c | 10 ++++ 3 files changed, 137 insertions(+), 2 deletions(-) diff --git a/include/linux/damon.h b/include/linux/damon.h index ee2d0879c292..6df716533fbf 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -191,6 +191,8 @@ enum damos_action { * @DAMOS_QUOTA_NODE_MEM_FREE_BP: MemFree ratio of a node. * @DAMOS_QUOTA_NODE_MEMCG_USED_BP: MemUsed ratio of a node for a cgroup. * @DAMOS_QUOTA_NODE_MEMCG_FREE_BP: MemFree ratio of a node for a cgroup. + * @DAMOS_QUOTA_NODE_ELIGIBLE_MEM_BP: Scheme-eligible memory ratio of a node. + * @DAMOS_QUOTA_NODE_INELIGIBLE_MEM_BP: Scheme-ineligible memory ratio of a node. * @DAMOS_QUOTA_ACTIVE_MEM_BP: Active to total LRU memory ratio. * @DAMOS_QUOTA_INACTIVE_MEM_BP: Inactive to total LRU memory ratio. * @NR_DAMOS_QUOTA_GOAL_METRICS: Number of DAMOS quota goal metrics. @@ -204,6 +206,8 @@ enum damos_quota_goal_metric { DAMOS_QUOTA_NODE_MEM_FREE_BP, DAMOS_QUOTA_NODE_MEMCG_USED_BP, DAMOS_QUOTA_NODE_MEMCG_FREE_BP, + DAMOS_QUOTA_NODE_ELIGIBLE_MEM_BP, + DAMOS_QUOTA_NODE_INELIGIBLE_MEM_BP, DAMOS_QUOTA_ACTIVE_MEM_BP, DAMOS_QUOTA_INACTIVE_MEM_BP, NR_DAMOS_QUOTA_GOAL_METRICS, @@ -555,6 +559,7 @@ struct damos_migrate_dests { * @ops_filters: ops layer handling &struct damos_filter objects list. * @last_applied: Last @action applied ops-managing entity. * @stat: Statistics of this scheme. + * @eligible_bytes_per_node: Scheme-eligible bytes per NUMA node. * @max_nr_snapshots: Upper limit of nr_snapshots stat. * @list: List head for siblings. * @@ -644,6 +649,7 @@ struct damos { struct list_head ops_filters; void *last_applied; struct damos_stat stat; + unsigned long eligible_bytes_per_node[MAX_NUMNODES]; unsigned long max_nr_snapshots; struct list_head list; }; diff --git a/mm/damon/core.c b/mm/damon/core.c index b438355ab54a..3e1cb850f067 100644 --- a/mm/damon/core.c +++ b/mm/damon/core.c @@ -2544,6 +2544,111 @@ static unsigned long damos_get_node_memcg_used_bp( } #endif +#ifdef CONFIG_NUMA +/* + * damos_scheme_uses_eligible_metrics() - Check if scheme uses eligible metrics. + * @s: The scheme + * + * Returns true if any quota goal uses node_eligible_mem_bp or + * node_ineligible_mem_bp metrics, which require eligible bytes calculation. + */ +static bool damos_scheme_uses_eligible_metrics(struct damos *s) +{ + struct damos_quota_goal *goal; + struct damos_quota *quota = &s->quota; + + damos_for_each_quota_goal(goal, quota) { + if (goal->metric == DAMOS_QUOTA_NODE_ELIGIBLE_MEM_BP || + goal->metric == DAMOS_QUOTA_NODE_INELIGIBLE_MEM_BP) + return true; + } + return false; +} + +/* + * damos_calc_eligible_bytes_per_node() - Calculate eligible bytes per node. + * @c: The DAMON context + * @s: The scheme + * + * Calculates scheme-eligible bytes per NUMA node based on access pattern + * matching. A region is eligible if it matches the scheme's access pattern + * (size, nr_accesses, age). + */ +static void damos_calc_eligible_bytes_per_node(struct damon_ctx *c, + struct damos *s) +{ + struct damon_target *t; + struct damon_region *r; + phys_addr_t paddr; + int nid; + + memset(s->eligible_bytes_per_node, 0, + sizeof(s->eligible_bytes_per_node)); + + damon_for_each_target(t, c) { + damon_for_each_region(r, t) { + if (!__damos_valid_target(r, s)) + continue; + paddr = (phys_addr_t)r->ar.start * c->addr_unit; + nid = pfn_to_nid(PHYS_PFN(paddr)); + if (nid >= 0 && nid < MAX_NUMNODES) + s->eligible_bytes_per_node[nid] += + damon_sz_region(r) * c->addr_unit; + } + } +} + +static unsigned long damos_get_node_eligible_mem_bp(struct damos *s, int nid) +{ + unsigned long total_eligible = 0; + unsigned long node_eligible; + int n; + + if (nid < 0 || nid >= MAX_NUMNODES) + return 0; + + for_each_online_node(n) + total_eligible += s->eligible_bytes_per_node[n]; + + if (!total_eligible) + return 0; + + node_eligible = s->eligible_bytes_per_node[nid]; + + return mult_frac(node_eligible, 10000, total_eligible); +} + +static unsigned long damos_get_node_ineligible_mem_bp(struct damos *s, int nid) +{ + unsigned long eligible_bp = damos_get_node_eligible_mem_bp(s, nid); + + if (eligible_bp == 0) + return 10000; + + return 10000 - eligible_bp; +} +#else +static bool damos_scheme_uses_eligible_metrics(struct damos *s) +{ + return false; +} + +static void damos_calc_eligible_bytes_per_node(struct damon_ctx *c, + struct damos *s) +{ +} + +static unsigned long damos_get_node_eligible_mem_bp(struct damos *s, int nid) +{ + return 0; +} + +static unsigned long damos_get_node_ineligible_mem_bp(struct damos *s, int nid) +{ + return 0; +} +#endif + /* * Returns LRU-active or inactive memory to total LRU memory size ratio. */ @@ -2562,7 +2667,8 @@ static unsigned int damos_get_in_active_mem_bp(bool active_ratio) return mult_frac(inactive, 10000, total); } -static void damos_set_quota_goal_current_value(struct damos_quota_goal *goal) +static void damos_set_quota_goal_current_value(struct damos_quota_goal *goal, + struct damos *s) { u64 now_psi_total; @@ -2584,6 +2690,14 @@ static void damos_set_quota_goal_current_value(struct damos_quota_goal *goal) case DAMOS_QUOTA_NODE_MEMCG_FREE_BP: goal->current_value = damos_get_node_memcg_used_bp(goal); break; + case DAMOS_QUOTA_NODE_ELIGIBLE_MEM_BP: + goal->current_value = damos_get_node_eligible_mem_bp(s, + goal->nid); + break; + case DAMOS_QUOTA_NODE_INELIGIBLE_MEM_BP: + goal->current_value = damos_get_node_ineligible_mem_bp(s, + goal->nid); + break; case DAMOS_QUOTA_ACTIVE_MEM_BP: case DAMOS_QUOTA_INACTIVE_MEM_BP: goal->current_value = damos_get_in_active_mem_bp( @@ -2597,11 +2711,12 @@ static void damos_set_quota_goal_current_value(struct damos_quota_goal *goal) /* Return the highest score since it makes schemes least aggressive */ static unsigned long damos_quota_score(struct damos_quota *quota) { + struct damos *s = container_of(quota, struct damos, quota); struct damos_quota_goal *goal; unsigned long highest_score = 0; damos_for_each_quota_goal(goal, quota) { - damos_set_quota_goal_current_value(goal); + damos_set_quota_goal_current_value(goal, s); highest_score = max(highest_score, mult_frac(goal->current_value, 10000, goal->target_value)); @@ -2693,6 +2808,10 @@ static void damos_adjust_quota(struct damon_ctx *c, struct damos *s) if (!quota->ms && !quota->sz && list_empty("a->goals)) return; + /* Calculate eligible bytes per node for quota goal metrics */ + if (damos_scheme_uses_eligible_metrics(s)) + damos_calc_eligible_bytes_per_node(c, s); + /* First charge window */ if (!quota->total_charged_sz && !quota->charged_from) { quota->charged_from = jiffies; diff --git a/mm/damon/sysfs-schemes.c b/mm/damon/sysfs-schemes.c index fe2e3b2db9e1..232b33f5cbfb 100644 --- a/mm/damon/sysfs-schemes.c +++ b/mm/damon/sysfs-schemes.c @@ -1079,6 +1079,14 @@ struct damos_sysfs_qgoal_metric_name damos_sysfs_qgoal_metric_names[] = { .metric = DAMOS_QUOTA_NODE_MEMCG_FREE_BP, .name = "node_memcg_free_bp", }, + { + .metric = DAMOS_QUOTA_NODE_ELIGIBLE_MEM_BP, + .name = "node_eligible_mem_bp", + }, + { + .metric = DAMOS_QUOTA_NODE_INELIGIBLE_MEM_BP, + .name = "node_ineligible_mem_bp", + }, { .metric = DAMOS_QUOTA_ACTIVE_MEM_BP, .name = "active_mem_bp", @@ -2669,6 +2677,8 @@ static int damos_sysfs_add_quota_score( break; case DAMOS_QUOTA_NODE_MEM_USED_BP: case DAMOS_QUOTA_NODE_MEM_FREE_BP: + case DAMOS_QUOTA_NODE_ELIGIBLE_MEM_BP: + case DAMOS_QUOTA_NODE_INELIGIBLE_MEM_BP: goal->nid = sysfs_goal->nid; break; case DAMOS_QUOTA_NODE_MEMCG_USED_BP: -- 2.43.0