* [PATCH v4 0/3] mm/damon: Introduce a huge page collapsing mechanism using auto tuning
@ 2026-08-31 14:47 SJ Park
2026-08-31 14:47 ` [PATCH v4 1/3] mm/damon: Introduce DAMOS_QUOTA_HUGEPAGE " SJ Park
` (3 more replies)
0 siblings, 4 replies; 17+ messages in thread
From: SJ Park @ 2026-08-31 14:47 UTC (permalink / raw)
To: Andrew Morton
Cc: SJ Park, Liam R. Howlett, David Hildenbrand, Jonathan Corbet,
Lorenzo Stoakes, Michal Hocko, Mike Rapoport, Randy Dunlap,
Shuah Khan, Suren Baghdasaryan, Vlastimil Babka, damon, linux-doc,
linux-kernel, linux-mm
From: Asier Gutierrez <gutierrez.asier@huawei-partners.com>
Overview
========
This patch set introduces a new autotuning which allows to collapse hot
regions into hugepages.
Motivation
==========
Since TLB is a bottleneck for many systems[1], a way to optimize TLB
misses (or hits) is to use huge pages. Unfortunately, using "always" in
THP leads to memory fragmentation and memory waste. For this reason,
most application guides and system administrators suggest to disable
THP.
Selective huge page collapse per process is possible using prctl and a
launcher. However, this does not solve the issue with hot region
detection. Additionally, it the sysadmin should create a launcher that
uses PRCTL to enable THP for a particular process.
We can use the DAMON support for DAMOS_HUGEPAGE and DAMOS_COLLAPSE, to
target a certain process. DAMOS_COLLAPSE can also target the hot regions
in that process.
Still, there is an issue with the amount of huge page consumption. Since
huge pages can lead to memory fragmentation and waste, there should be a
way to limit the amount of huge page consumption. There is hugetlbfs,
but it requires changes to the application code or the use of
libhugetlbfs.
DAMON has now a way to autotune some of the variables and adjust quotas
automatically, so that DAMON is fired only under the right
circumstances. It would be nice to have something similar, but for huge
pages.
Solution
========
A new autotuning quota goal[2], damos_hugepage_mem_bp, is introduced,
which checks the huge page consumption to total memory consumption. This
new quota mechanism reuses current autotuning architecture.
In order to test this new mechanism, a sample module[3] was created, but
not included in this patch series. To demonstrate the tool, damo user
space tool was modified[4], which sets up huge pages collapse
autotuning.
Benchmarks
==========
Setup: physical server with arm64 processor with 4 NUMA nodes, 1 TB RAM
and running mariaDB 10.5.29. Sysbench was used for the benchmark, with
20 tables and 3 million rows per table. The database was pinned to one
of the nodes, and the benchmark framework to a different node. No
network traffic involved in the benchmark.
Damo user space tool was forked and hugepage_mem_bp support added[4].
DAMON was lauched using this command line:
sudo ./damo start $(pidof mariadbd) \
--monitoring_nr_regions_range 10 1000 \
--monitoring_intervals 5000 100000 60000000 \
--damos_quota_time 0 --damos_quota_space 128000000 \
--damos_quota_interval 1000 \
--damos_quota_weights 0 1 1 \
--damos_quota_goal hugepage_mem_bp <target> \
--damos_quota_goal_tuner temporal \
--damos_apply_interval 50000 \
--damos_access_rate 0 max --damos_age 50 max \
--damos_action collapse --debug_damon
<target> was 1000 to taget 10% hugepage to total memory ratio, or 2500
to target 25%. Tuner was also tested with consistent and temporal.
Results
=======
After the last timestamp, there was no change in huge page use, and the
total huge page to memory consumption ratio barely moved.
hugepage_mem_bp: 1000
goal tuner: temporal
+-----------+----------------+----------------+----------------------+
| timestamp | total mem used | huge page used | percentage hugepage |
+-----------+----------------+----------------+----------------------+
| 0 | 16945.04297 | 0 | 0 |
| 7 | 17008.69531 | 74 | 0.435071583 |
| 8 | 17036.40234 | 194 | 1.138738074 |
| 9 | 17017.01563 | 314 | 1.845211916 |
| 10 | 17029.67969 | 434 | 2.548491856 |
| 61 | 17111.30859 | 584 | 3.412947623 |
| 120 | 17071.05859 | 694 | 4.065360072 |
| 180 | 17133.88281 | 804 | 4.692456513 |
| 203 | 17088.16406 | 916 | 5.360435426 |
| 204 | 17126.34766 | 1046 | 6.107548562 |
| 205 | 17093.84375 | 1176 | 6.879669764 |
| 206 | 17142.77734 | 1298 | 7.571701913 |
| 209 | 17149.17969 | 1686 | 9.831374041 |
| 210 | 17097.30859 | 1754 | 10.25892462 |
+-----------+----------------+----------------+----------------------+
hugepage_mem_bp: 1000
goal tuner: consistent
+-----------+----------------+----------------+----------------------+
| timestamp | total mem used | huge page used | percentage hugepage |
+-----------+----------------+----------------+----------------------+
| 0 | 16955.24609 | 0 | 0 |
| 34 | 17039.71875 | 106 | 0.622075995 |
| 78 | 17009.47656 | 554 | 3.257007927 |
| 90 | 17048.92188 | 596 | 3.495822225 |
| 150 | 17092.90625 | 706 | 4.130368409 |
| 180 | 17053.08984 | 764 | 4.480126517 |
| 233 | 17100.50391 | 1496 | 8.748280216 |
| 239 | 17098.89063 | 2216 | 12.95990511 |
| 240 | 17135.44531 | 2334 | 13.62088908 |
| 245 | 17132.55078 | 2932 | 17.11362212 |
| 246 | 17117.95313 | 3052 | 17.82923448 |
| 250 | 17163.12109 | 3532 | 20.57900763 |
+-----------+----------------+----------------+----------------------+
hugepage_mem_bp: 2500
goal tuner: temporal
+-----------+----------------+----------------+----------------------+
| timestamp | total mem used | huge page used | percentage hugepage |
+-----------+----------------+----------------+----------------------+
| 0 | 17010.31641 | 0 | 0 |
| 9 | 17063.6875 | 50 | 0.2930199 |
| 10 | 17051.75781 | 170 | 0.996964664 |
| 60 | 17133.85547 | 572 | 3.338419663 |
| 90 | 17192.07813 | 626 | 3.641211932 |
| 120 | 17221.44531 | 682 | 3.960178647 |
| 181 | 17199.76172 | 790 | 4.593086886 |
| 208 | 17222.77734 | 1206 | 7.002354939 |
| 214 | 17245.17969 | 1904 | 11.04076637 |
| 215 | 17240.45703 | 2024 | 11.73982799 |
| 220 | 17234.79688 | 2624 | 15.22501262 |
| 228 | 17222.83594 | 3584 | 20.80958103 |
| 231 | 17247.55469 | 3944 | 22.86700968 |
| 235 | 17229.37109 | 4424 | 25.67708349 |
+-----------+----------------+----------------+----------------------+
hugepage_mem_bp: 1000
goal tuner: consist
+-----------+----------------+----------------+----------------------+
| timestamp | total mem used | huge page used | percentage hugepage |
+-----------+----------------+----------------+----------------------+
| 0 | 17125.85156 | 0 | 0 |
| 38 | 17081.23438 | 76 | 0.444932716 |
| 39 | 17133.11719 | 196 | 1.143983304 |
| 40 | 17119.83984 | 316 | 1.84581166 |
| 60 | 17109.72656 | 554 | 3.237924335 |
| 90 | 17164.11328 | 628 | 3.65879664 |
| 180 | 17177.66016 | 792 | 4.610639591 |
| 220 | 17180.86719 | 1378 | 8.020549749 |
| 226 | 17187.82031 | 1980 | 11.51978531 |
| 233 | 17143.48438 | 2818 | 16.4377319 |
| 240 | 17137.38281 | 3656 | 21.33347921 |
| 250 | 17175.5 | 4856 | 28.27283049 |
| 260 | 17199.66406 | 6056 | 35.20999002 |
| 270 | 17203.98438 | 7254 | 42.16465118 |
| 275 | 17207.21875 | 7762 | 45.10897498 |
+-----------+----------------+----------------+----------------------+
More detailed tables are provided here[5]
From this, we can conclude that the huge page autotuner works fine,
achieving the target. When using consistent autotuner, it actually
over-achieves the target, which is expected, since quota esz_bp is not
set to 0 to cap the DAMOS policy.
Patches Sequence
================
Patch 1 -> Introduce DAMOS_QUOTA_HUGEPAGE_MEM_BP and autotuning
Patch 2 -> sysfs support for the new quota goal
Patch 3 -> Document hugepage_mem_bp parameter
[1] https://dl.acm.org/doi/pdf/10.1145/3307650.3322227
[2] https://lore.kernel.org/e67f05ad-dbb9-45e6-ba30-b167a99ac67d@huawei-partners.com
[3] https://lore.kernel.org/20260616150316.580819-3-gutierrez.asier@huawei-partners.com
[4] https://github.com/asierHuawei/damo/commit/79ae1a4ab1c012a7161db85a000d14f08fa36736
[5] https://lore.kernel.org/all/03f678dd-9ef3-4b97-b753-c2e4554c5159@huawei-partners.com/
Changes from previous versions
==============================
v3 -> v4
- v3: https://lore.kernel.org/20260720120140.881468-1-gutierrez.asier@huawei-partners.com
- Add R-b: from SJ.
- Handle a per-cpu count race where free pages larger than total
pages.
- Return 10,000 as hugepage memory ratio for the racy corner case,
instead of MAX_INT.
- Wordsmith commit message.
- Rebase to latest mm-new.
v2[6] -> v3
- Reworked the cover letter to make more clean the intents, design
choices and results.
- Added a guard in damos_hugepage_mem_bp in order to avoid potential
division by 0.[7]
- Fixed a typo in the documentation.
v1[8] -> v2
- Rebased onto updated mm-new
- Removed the sample module altogether, since it will increase
maintenance costs[9]
- Document hugepage_mem_bp in design.rst
RFC 4[10] -> v1
- Renamed config to SAMPLE_DAMON_HPAGE, file to hpage.c and functions
to damon_sample_hpage_...
- Make the module depend on TRANSPARENT_HUGEPAGE, since the module
will need some THP functions anyway
- Removed documentation, since this is just a sample module
- Removed DAMOS_QUOTA_HUGEPAGE_MEM_BP from damos_sysfs_add_quota_score
- Added a short description of the module in Kconfig
RFC 3[11] -> RFC 4
- Simplified the module
- Removed unnecessary parameters
- Renamed DAMOS_QUOTA_HUGEPAGE_MEM_BP to unify the naming style
- Switched to DAMOS_QUOTA_GOAL_TUNER_TEMPORAL
- Updated the documentation
- Removed new interface for context creation with DAMON_OPS_VADDR
RFC 2[12] -> RFC 3
- Module moved to samples
- Change autotune to monitor total memory and hugepage
- Added performnace benchmarks to the cover letter
- Bail out gracefully when trying to start disable the module after
the monitored task exited. This issue was discovered by sashiko [13]
- Fixed typos and added quota_sz to the documentation discovered by
sashiko [14]
RFC 1[15] -> RFC 2
- Rebased into mm-new
- Use DAMOS_COLLAPSE instead of DAMOS_HUGEPAGE
- Fixed an issue that returned silently an error when the PID didn't
exist in the system.[16]
[6] https://lore.kernel.org/all/20260714150116.382521-1-gutierrez.asier@huawei-partners.com
[7] https://lore.kernel.org/all/20260715151615.99767-1-sj@kernel.org/
[8] https://lore.kernel.org/20260616150316.580819-1-gutierrez.asier@huawei-partners.com
[9] https://lore.kernel.org/20260618150806.4633-1-sj@kernel.org
[10] https://lore.kernel.org/20260611150244.3454699-1-gutierrez.asier@huawei-partners.com
[11] https://lore.kernel.org/20260604150338.501128-1-gutierrez.asier@huawei-partners.com
[12] https://lore.kernel.org/20260522145518.158910-1-gutierrez.asier@huawei-partners.com
[13] https://lore.kernel.org/20260522171210.900B11F00A3D@smtp.kernel.org
[14] https://lore.kernel.org/20260522171633.AAF5B1F000E9@smtp.kernel.org
[15] https://lore.kernel.org/20260430134139.2446417-1-gutierrez.asier@huawei-partners.com
[16] https://lore.kernel.org/all/20260430154338.E22E6C2BCB3@smtp.kernel.org/
Note to Andrew:
Footnote references 6-16 are only used for changelog. Hence those can
be removed with changelog.
Asier Gutierrez (3):
mm/damon: Introduce DAMOS_QUOTA_HUGEPAGE auto tuning
mm/damon/sysfs: support hugepage_mem_bp quota goal metric
Docs/mm/damon/design: Document hugepage_mem_bp target metric
Documentation/mm/damon/design.rst | 2 ++
include/linux/damon.h | 2 ++
mm/damon/core.c | 19 +++++++++++++++++++
mm/damon/sysfs-schemes.c | 4 ++++
4 files changed, 27 insertions(+)
base-commit: 0ae786fcfff4cfea3290e16a65fe12f5c99a6d2f
--
2.47.3
^ permalink raw reply [flat|nested] 17+ messages in thread
* [PATCH v4 1/3] mm/damon: Introduce DAMOS_QUOTA_HUGEPAGE auto tuning
2026-08-31 14:47 [PATCH v4 0/3] mm/damon: Introduce a huge page collapsing mechanism using auto tuning SJ Park
@ 2026-08-31 14:47 ` SJ Park
2026-09-01 6:58 ` Lian Wang
2026-08-31 14:47 ` [PATCH v4 2/3] mm/damon/sysfs: support hugepage_mem_bp quota goal metric SJ Park
` (2 subsequent siblings)
3 siblings, 1 reply; 17+ messages in thread
From: SJ Park @ 2026-08-31 14:47 UTC (permalink / raw)
To: Andrew Morton; +Cc: Asier Gutierrez, SJ Park, damon, linux-kernel, linux-mm
From: Asier Gutierrez <gutierrez.asier@huawei-partners.com>
Introduce DAMOS_QUOTA_HUGEPAGE_MEM_BP auto tuning. Add a new DAMOS quota
goal metric to measure the amount of huge page consumption to total
memory consumption ratio.
Vmstat may lag, which in some cases may lead to NR_FREE_PAGES being
greater than or equal to the amount of RAM in the system. A guard is
added to avoid the extremely unlikely case [1]. In the case, return 100%
(10000 bp).
[1] https://lore.kernel.org/all/20260715151615.99767-1-sj@kernel.org/
Signed-off-by: Asier Gutierrez <gutierrez.asier@huawei-partners.com>
Reviewed-by: SJ Park <sj@kernel.org>
Signed-off-by: SJ Park <sj@kernel.org>
---
include/linux/damon.h | 2 ++
mm/damon/core.c | 19 +++++++++++++++++++
2 files changed, 21 insertions(+)
diff --git a/include/linux/damon.h b/include/linux/damon.h
index 5607f98ec6306..310509455db13 100644
--- a/include/linux/damon.h
+++ b/include/linux/damon.h
@@ -153,6 +153,7 @@ enum damos_action {
* @DAMOS_QUOTA_INACTIVE_MEM_BP: Inactive to total LRU memory ratio.
* @DAMOS_QUOTA_NODE_ELIGIBLE_MEM_BP: Scheme-eligible memory ratio of a
* node in basis points (0-10000).
+ * @DAMOS_QUOTA_HUGEPAGE_MEM_BP: Huge page to total used memory ratio.
* @NR_DAMOS_QUOTA_GOAL_METRICS: Number of DAMOS quota goal metrics.
*
* Metrics equal to larger than @NR_DAMOS_QUOTA_GOAL_METRICS are unsupported.
@@ -167,6 +168,7 @@ enum damos_quota_goal_metric {
DAMOS_QUOTA_ACTIVE_MEM_BP,
DAMOS_QUOTA_INACTIVE_MEM_BP,
DAMOS_QUOTA_NODE_ELIGIBLE_MEM_BP,
+ DAMOS_QUOTA_HUGEPAGE_MEM_BP,
NR_DAMOS_QUOTA_GOAL_METRICS,
};
diff --git a/mm/damon/core.c b/mm/damon/core.c
index a5ea3e61e7f41..3fa1b096c8330 100644
--- a/mm/damon/core.c
+++ b/mm/damon/core.c
@@ -3038,6 +3038,22 @@ static unsigned int damos_get_in_active_mem_bp(bool active_ratio)
return mult_frac(inactive, 10000, total);
}
+static unsigned int damos_hugepage_mem_bp(void)
+{
+ unsigned long thp, total_pages, free_pages;
+
+ total_pages = totalram_pages();
+ free_pages = global_zone_page_state(NR_FREE_PAGES);
+
+ if (total_pages <= free_pages)
+ return 10000;
+
+ thp = global_node_page_state(NR_ANON_THPS) +
+ global_node_page_state(NR_SHMEM_THPS) +
+ global_node_page_state(NR_FILE_THPS);
+ return mult_frac(thp, 10000, total_pages - free_pages);
+}
+
static void damos_set_quota_goal_current_value(struct damon_ctx *c,
struct damos *s, struct damos_quota_goal *goal)
{
@@ -3074,6 +3090,9 @@ static void damos_set_quota_goal_current_value(struct damon_ctx *c,
goal->current_value = damos_get_node_eligible_mem_bp(c, s,
goal->nid);
break;
+ case DAMOS_QUOTA_HUGEPAGE_MEM_BP:
+ goal->current_value = damos_hugepage_mem_bp();
+ break;
default:
break;
}
--
2.47.3
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v4 2/3] mm/damon/sysfs: support hugepage_mem_bp quota goal metric
2026-08-31 14:47 [PATCH v4 0/3] mm/damon: Introduce a huge page collapsing mechanism using auto tuning SJ Park
2026-08-31 14:47 ` [PATCH v4 1/3] mm/damon: Introduce DAMOS_QUOTA_HUGEPAGE " SJ Park
@ 2026-08-31 14:47 ` SJ Park
2026-08-31 14:47 ` [PATCH v4 3/3] Docs/mm/damon/design: Document hugepage_mem_bp target metric SJ Park
2026-09-01 0:49 ` [PATCH v4 0/3] mm/damon: Introduce a huge page collapsing mechanism using auto tuning SJ Park
3 siblings, 0 replies; 17+ messages in thread
From: SJ Park @ 2026-08-31 14:47 UTC (permalink / raw)
To: Andrew Morton; +Cc: Asier Gutierrez, SJ Park, damon, linux-kernel, linux-mm
From: Asier Gutierrez <gutierrez.asier@huawei-partners.com>
DAMOS has a new autotune policy metric: DAMOS_QUOTA_HUGEPAGE_MEM_BP.
This patch exposes DAMOS_QUOTA_HUGEPAGE_MEM_BP through sysfs.
Add the "hugepage_mem_bp" to the sysfs-schemes interface.
Signed-off-by: Asier Gutierrez <gutierrez.asier@huawei-partners.com>
Reviewed-by: SJ Park <sj@kernel.org>
Signed-off-by: SJ Park <sj@kernel.org>
---
mm/damon/sysfs-schemes.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/mm/damon/sysfs-schemes.c b/mm/damon/sysfs-schemes.c
index 32f495a96b17a..d9b81d7b5910e 100644
--- a/mm/damon/sysfs-schemes.c
+++ b/mm/damon/sysfs-schemes.c
@@ -1269,6 +1269,10 @@ struct damos_sysfs_qgoal_metric_name damos_sysfs_qgoal_metric_names[] = {
.metric = DAMOS_QUOTA_NODE_ELIGIBLE_MEM_BP,
.name = "node_eligible_mem_bp",
},
+ {
+ .metric = DAMOS_QUOTA_HUGEPAGE_MEM_BP,
+ .name = "hugepage_mem_bp",
+ },
};
static ssize_t target_metric_show(struct kobject *kobj,
--
2.47.3
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v4 3/3] Docs/mm/damon/design: Document hugepage_mem_bp target metric
2026-08-31 14:47 [PATCH v4 0/3] mm/damon: Introduce a huge page collapsing mechanism using auto tuning SJ Park
2026-08-31 14:47 ` [PATCH v4 1/3] mm/damon: Introduce DAMOS_QUOTA_HUGEPAGE " SJ Park
2026-08-31 14:47 ` [PATCH v4 2/3] mm/damon/sysfs: support hugepage_mem_bp quota goal metric SJ Park
@ 2026-08-31 14:47 ` SJ Park
2026-08-31 15:45 ` Randy Dunlap
2026-09-01 0:49 ` [PATCH v4 0/3] mm/damon: Introduce a huge page collapsing mechanism using auto tuning SJ Park
3 siblings, 1 reply; 17+ messages in thread
From: SJ Park @ 2026-08-31 14:47 UTC (permalink / raw)
To: Andrew Morton
Cc: Asier Gutierrez, Liam R. Howlett, David Hildenbrand,
Jonathan Corbet, Lorenzo Stoakes, Michal Hocko, Mike Rapoport,
Randy Dunlap, SJ Park, Shuah Khan, Suren Baghdasaryan,
Vlastimil Babka, damon, linux-doc, linux-kernel, linux-mm
From: Asier Gutierrez <gutierrez.asier@huawei-partners.com>
Document hugepage_mem_bp metric exposed by sysfs.
Signed-off-by: Asier Gutierrez <gutierrez.asier@huawei-partners.com>
Reviewed-by: SJ Park <sj@kernel.org>
Signed-off-by: SJ Park <sj@kernel.org>
---
Documentation/mm/damon/design.rst | 2 ++
1 file changed, 2 insertions(+)
diff --git a/Documentation/mm/damon/design.rst b/Documentation/mm/damon/design.rst
index aed6cb1cf4831..a8c163475ef2b 100644
--- a/Documentation/mm/damon/design.rst
+++ b/Documentation/mm/damon/design.rst
@@ -713,6 +713,8 @@ mechanism tries to make ``current_value`` of ``target_metric`` be same to
bp (1/10,000).
- ``node_eligible_mem_bp``: Scheme target access pattern-eligible memory ratio
of a node in bp (1/10,000).
+- ``hugepage_mem_bp``: Total huge page to total used memory ratio in bp
+ (1/10,000).
``nid`` is optionally required for ``node_mem_used_bp``, ``node_mem_free_bp``,
``node_memcg_used_bp``, ``node_memcg_free_bp`` and ``node_eligible_mem_bp`` to
--
2.47.3
^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: [PATCH v4 3/3] Docs/mm/damon/design: Document hugepage_mem_bp target metric
2026-08-31 14:47 ` [PATCH v4 3/3] Docs/mm/damon/design: Document hugepage_mem_bp target metric SJ Park
@ 2026-08-31 15:45 ` Randy Dunlap
2026-08-31 20:00 ` Randy Dunlap
0 siblings, 1 reply; 17+ messages in thread
From: Randy Dunlap @ 2026-08-31 15:45 UTC (permalink / raw)
To: SJ Park, Andrew Morton
Cc: Asier Gutierrez, Liam R. Howlett, David Hildenbrand,
Jonathan Corbet, Lorenzo Stoakes, Michal Hocko, Mike Rapoport,
Shuah Khan, Suren Baghdasaryan, Vlastimil Babka, damon, linux-doc,
linux-kernel, linux-mm
Hi,
On 8/31/26 7:47 AM, SJ Park wrote:
> From: Asier Gutierrez <gutierrez.asier@huawei-partners.com>
>
> Document hugepage_mem_bp metric exposed by sysfs.
>
> Signed-off-by: Asier Gutierrez <gutierrez.asier@huawei-partners.com>
> Reviewed-by: SJ Park <sj@kernel.org>
> Signed-off-by: SJ Park <sj@kernel.org>
> ---
> Documentation/mm/damon/design.rst | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/Documentation/mm/damon/design.rst b/Documentation/mm/damon/design.rst
> index aed6cb1cf4831..a8c163475ef2b 100644
> --- a/Documentation/mm/damon/design.rst
> +++ b/Documentation/mm/damon/design.rst
> @@ -713,6 +713,8 @@ mechanism tries to make ``current_value`` of ``target_metric`` be same to
> bp (1/10,000).
> - ``node_eligible_mem_bp``: Scheme target access pattern-eligible memory ratio
> of a node in bp (1/10,000).
> +- ``hugepage_mem_bp``: Total huge page to total used memory ratio in bp
> + (1/10,000).
>
> ``nid`` is optionally required for ``node_mem_used_bp``, ``node_mem_free_bp``,
> ``node_memcg_used_bp``, ``node_memcg_free_bp`` and ``node_eligible_mem_bp`` to
I've looked thru the entire design.rst file and cannot find anything telling
me (or anyone) what "bp" means. Explain??
thanks.
--
~Randy
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v4 3/3] Docs/mm/damon/design: Document hugepage_mem_bp target metric
2026-08-31 15:45 ` Randy Dunlap
@ 2026-08-31 20:00 ` Randy Dunlap
2026-09-01 0:47 ` SJ Park
0 siblings, 1 reply; 17+ messages in thread
From: Randy Dunlap @ 2026-08-31 20:00 UTC (permalink / raw)
To: SJ Park, Andrew Morton
Cc: Asier Gutierrez, Liam R. Howlett, David Hildenbrand,
Jonathan Corbet, Lorenzo Stoakes, Michal Hocko, Mike Rapoport,
Shuah Khan, Suren Baghdasaryan, Vlastimil Babka, damon, linux-doc,
linux-kernel, linux-mm
On 8/31/26 8:45 AM, Randy Dunlap wrote:
> Hi,
>
> On 8/31/26 7:47 AM, SJ Park wrote:
>> From: Asier Gutierrez <gutierrez.asier@huawei-partners.com>
>>
>> Document hugepage_mem_bp metric exposed by sysfs.
>>
>> Signed-off-by: Asier Gutierrez <gutierrez.asier@huawei-partners.com>
>> Reviewed-by: SJ Park <sj@kernel.org>
>> Signed-off-by: SJ Park <sj@kernel.org>
>> ---
>> Documentation/mm/damon/design.rst | 2 ++
>> 1 file changed, 2 insertions(+)
>>
>> diff --git a/Documentation/mm/damon/design.rst b/Documentation/mm/damon/design.rst
>> index aed6cb1cf4831..a8c163475ef2b 100644
>> --- a/Documentation/mm/damon/design.rst
>> +++ b/Documentation/mm/damon/design.rst
>> @@ -713,6 +713,8 @@ mechanism tries to make ``current_value`` of ``target_metric`` be same to
>> bp (1/10,000).
>> - ``node_eligible_mem_bp``: Scheme target access pattern-eligible memory ratio
>> of a node in bp (1/10,000).
>> +- ``hugepage_mem_bp``: Total huge page to total used memory ratio in bp
>> + (1/10,000).
>>
>> ``nid`` is optionally required for ``node_mem_used_bp``, ``node_mem_free_bp``,
>> ``node_memcg_used_bp``, ``node_memcg_free_bp`` and ``node_eligible_mem_bp`` to
>
> I've looked thru the entire design.rst file and cannot find anything telling
> me (or anyone) what "bp" means. Explain??
OK, I found it on the internet. I still think that it would be a
good idea to say "basis point" one time (the first time that it is used).
--
~Randy
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v4 3/3] Docs/mm/damon/design: Document hugepage_mem_bp target metric
2026-08-31 20:00 ` Randy Dunlap
@ 2026-09-01 0:47 ` SJ Park
2026-09-01 5:20 ` Gutierrez Asier
0 siblings, 1 reply; 17+ messages in thread
From: SJ Park @ 2026-09-01 0:47 UTC (permalink / raw)
To: Randy Dunlap
Cc: SJ Park, Andrew Morton, Asier Gutierrez, Liam R. Howlett,
David Hildenbrand, Jonathan Corbet, Lorenzo Stoakes, Michal Hocko,
Mike Rapoport, Shuah Khan, Suren Baghdasaryan, Vlastimil Babka,
damon, linux-doc, linux-kernel, linux-mm
On Mon, 31 Aug 2026 13:00:26 -0700 Randy Dunlap <rdunlap@infradead.org> wrote:
>
>
> On 8/31/26 8:45 AM, Randy Dunlap wrote:
> > Hi,
> >
> > On 8/31/26 7:47 AM, SJ Park wrote:
> >> From: Asier Gutierrez <gutierrez.asier@huawei-partners.com>
> >>
> >> Document hugepage_mem_bp metric exposed by sysfs.
> >>
> >> Signed-off-by: Asier Gutierrez <gutierrez.asier@huawei-partners.com>
> >> Reviewed-by: SJ Park <sj@kernel.org>
> >> Signed-off-by: SJ Park <sj@kernel.org>
> >> ---
> >> Documentation/mm/damon/design.rst | 2 ++
> >> 1 file changed, 2 insertions(+)
> >>
> >> diff --git a/Documentation/mm/damon/design.rst b/Documentation/mm/damon/design.rst
> >> index aed6cb1cf4831..a8c163475ef2b 100644
> >> --- a/Documentation/mm/damon/design.rst
> >> +++ b/Documentation/mm/damon/design.rst
> >> @@ -713,6 +713,8 @@ mechanism tries to make ``current_value`` of ``target_metric`` be same to
> >> bp (1/10,000).
> >> - ``node_eligible_mem_bp``: Scheme target access pattern-eligible memory ratio
> >> of a node in bp (1/10,000).
> >> +- ``hugepage_mem_bp``: Total huge page to total used memory ratio in bp
> >> + (1/10,000).
> >>
> >> ``nid`` is optionally required for ``node_mem_used_bp``, ``node_mem_free_bp``,
> >> ``node_memcg_used_bp``, ``node_memcg_free_bp`` and ``node_eligible_mem_bp`` to
> >
> > I've looked thru the entire design.rst file and cannot find anything telling
> > me (or anyone) what "bp" means. Explain??
>
> OK, I found it on the internet. I still think that it would be a
> good idea to say "basis point" one time (the first time that it is used).
You are right, and I agree. I will add that clarification before the next
merge window.
>
>
> --
> ~Randy
Thanks,
SJ
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v4 0/3] mm/damon: Introduce a huge page collapsing mechanism using auto tuning
2026-08-31 14:47 [PATCH v4 0/3] mm/damon: Introduce a huge page collapsing mechanism using auto tuning SJ Park
` (2 preceding siblings ...)
2026-08-31 14:47 ` [PATCH v4 3/3] Docs/mm/damon/design: Document hugepage_mem_bp target metric SJ Park
@ 2026-09-01 0:49 ` SJ Park
3 siblings, 0 replies; 17+ messages in thread
From: SJ Park @ 2026-09-01 0:49 UTC (permalink / raw)
To: SJ Park
Cc: Andrew Morton, Liam R. Howlett, David Hildenbrand,
Jonathan Corbet, Lorenzo Stoakes, Michal Hocko, Mike Rapoport,
Randy Dunlap, Shuah Khan, Suren Baghdasaryan, Vlastimil Babka,
damon, linux-doc, linux-kernel, linux-mm
On Mon, 31 Aug 2026 07:47:27 -0700 SJ Park <sj@kernel.org> wrote:
> From: Asier Gutierrez <gutierrez.asier@huawei-partners.com>
>
> Overview
> ========
> This patch set introduces a new autotuning which allows to collapse hot
> regions into hugepages.
Sashiko found no blocker for this series. Sashiko sent findings to damon@
mailing list [1], and I replied to all the comments. Please read those for
details.
[1] https://lore.kernel.org/damon/
Thanks,
SJ
[...]
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v4 3/3] Docs/mm/damon/design: Document hugepage_mem_bp target metric
2026-09-01 0:47 ` SJ Park
@ 2026-09-01 5:20 ` Gutierrez Asier
2026-09-01 5:31 ` SJ Park
0 siblings, 1 reply; 17+ messages in thread
From: Gutierrez Asier @ 2026-09-01 5:20 UTC (permalink / raw)
To: SJ Park, Randy Dunlap
Cc: Andrew Morton, Liam R. Howlett, David Hildenbrand,
Jonathan Corbet, Lorenzo Stoakes, Michal Hocko, Mike Rapoport,
Shuah Khan, Suren Baghdasaryan, Vlastimil Babka, damon, linux-doc,
linux-kernel, linux-mm
On 9/1/2026 3:47 AM, SJ Park wrote:
> On Mon, 31 Aug 2026 13:00:26 -0700 Randy Dunlap <rdunlap@infradead.org> wrote:
Hi,
>
>>
>>
>> On 8/31/26 8:45 AM, Randy Dunlap wrote:
>>> Hi,
>>>
>>> On 8/31/26 7:47 AM, SJ Park wrote:
>>>> From: Asier Gutierrez <gutierrez.asier@huawei-partners.com>
>>>>
>>>> Document hugepage_mem_bp metric exposed by sysfs.
>>>>
>>>> Signed-off-by: Asier Gutierrez <gutierrez.asier@huawei-partners.com>
>>>> Reviewed-by: SJ Park <sj@kernel.org>
>>>> Signed-off-by: SJ Park <sj@kernel.org>
>>>> ---
>>>> Documentation/mm/damon/design.rst | 2 ++
>>>> 1 file changed, 2 insertions(+)
>>>>
>>>> diff --git a/Documentation/mm/damon/design.rst b/Documentation/mm/damon/design.rst
>>>> index aed6cb1cf4831..a8c163475ef2b 100644
>>>> --- a/Documentation/mm/damon/design.rst
>>>> +++ b/Documentation/mm/damon/design.rst
>>>> @@ -713,6 +713,8 @@ mechanism tries to make ``current_value`` of ``target_metric`` be same to
>>>> bp (1/10,000).
>>>> - ``node_eligible_mem_bp``: Scheme target access pattern-eligible memory ratio
>>>> of a node in bp (1/10,000).
>>>> +- ``hugepage_mem_bp``: Total huge page to total used memory ratio in bp
>>>> + (1/10,000).
>>>>
>>>> ``nid`` is optionally required for ``node_mem_used_bp``, ``node_mem_free_bp``,
>>>> ``node_memcg_used_bp``, ``node_memcg_free_bp`` and ``node_eligible_mem_bp`` to
>>>
>>> I've looked thru the entire design.rst file and cannot find anything telling
>>> me (or anyone) what "bp" means. Explain??
>>
>> OK, I found it on the internet. I still think that it would be a
>> good idea to say "basis point" one time (the first time that it is used).
>
> You are right, and I agree. I will add that clarification before the next
> merge window.
>
>>
>>
>> --
>> ~Randy
>
>
> Thanks,
> SJ
SJ, should I submit a new patch to fix design.rst?
--
Asier Gutierrez
Huawei
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v4 3/3] Docs/mm/damon/design: Document hugepage_mem_bp target metric
2026-09-01 5:20 ` Gutierrez Asier
@ 2026-09-01 5:31 ` SJ Park
0 siblings, 0 replies; 17+ messages in thread
From: SJ Park @ 2026-09-01 5:31 UTC (permalink / raw)
To: Gutierrez Asier
Cc: SJ Park, Randy Dunlap, Andrew Morton, Liam R. Howlett,
David Hildenbrand, Jonathan Corbet, Lorenzo Stoakes, Michal Hocko,
Mike Rapoport, Shuah Khan, Suren Baghdasaryan, Vlastimil Babka,
damon, linux-doc, linux-kernel, linux-mm
On Tue, 1 Sep 2026 08:20:50 +0300 Gutierrez Asier <gutierrez.asier@huawei-partners.com> wrote:
> On 9/1/2026 3:47 AM, SJ Park wrote:
> > On Mon, 31 Aug 2026 13:00:26 -0700 Randy Dunlap <rdunlap@infradead.org> wrote:
> Hi,
> >
> >>
> >>
> >> On 8/31/26 8:45 AM, Randy Dunlap wrote:
> >>> Hi,
> >>>
> >>> On 8/31/26 7:47 AM, SJ Park wrote:
> >>>> From: Asier Gutierrez <gutierrez.asier@huawei-partners.com>
> >>>>
> >>>> Document hugepage_mem_bp metric exposed by sysfs.
> >>>>
> >>>> Signed-off-by: Asier Gutierrez <gutierrez.asier@huawei-partners.com>
> >>>> Reviewed-by: SJ Park <sj@kernel.org>
> >>>> Signed-off-by: SJ Park <sj@kernel.org>
> >>>> ---
> >>>> Documentation/mm/damon/design.rst | 2 ++
> >>>> 1 file changed, 2 insertions(+)
> >>>>
> >>>> diff --git a/Documentation/mm/damon/design.rst b/Documentation/mm/damon/design.rst
> >>>> index aed6cb1cf4831..a8c163475ef2b 100644
> >>>> --- a/Documentation/mm/damon/design.rst
> >>>> +++ b/Documentation/mm/damon/design.rst
> >>>> @@ -713,6 +713,8 @@ mechanism tries to make ``current_value`` of ``target_metric`` be same to
> >>>> bp (1/10,000).
> >>>> - ``node_eligible_mem_bp``: Scheme target access pattern-eligible memory ratio
> >>>> of a node in bp (1/10,000).
> >>>> +- ``hugepage_mem_bp``: Total huge page to total used memory ratio in bp
> >>>> + (1/10,000).
> >>>>
> >>>> ``nid`` is optionally required for ``node_mem_used_bp``, ``node_mem_free_bp``,
> >>>> ``node_memcg_used_bp``, ``node_memcg_free_bp`` and ``node_eligible_mem_bp`` to
> >>>
> >>> I've looked thru the entire design.rst file and cannot find anything telling
> >>> me (or anyone) what "bp" means. Explain??
> >>
> >> OK, I found it on the internet. I still think that it would be a
> >> good idea to say "basis point" one time (the first time that it is used).
> >
> > You are right, and I agree. I will add that clarification before the next
> > merge window.
> >
> >>
> >>
> >> --
> >> ~Randy
> >
> >
> > Thanks,
> > SJ
>
> SJ, should I submit a new patch to fix design.rst?
The lack of the clarification of 'bp' was introduced before this patch. So I
don't think you should do that on your own.
Of course you could do if you want to. But that should be a separate patch in
my opinion. Let me know if you will. Otherwise I will do.
Thanks,
SJ
[...]
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v4 1/3] mm/damon: Introduce DAMOS_QUOTA_HUGEPAGE auto tuning
2026-08-31 14:47 ` [PATCH v4 1/3] mm/damon: Introduce DAMOS_QUOTA_HUGEPAGE " SJ Park
@ 2026-09-01 6:58 ` Lian Wang
2026-09-01 14:23 ` SJ Park
0 siblings, 1 reply; 17+ messages in thread
From: Lian Wang @ 2026-09-01 6:58 UTC (permalink / raw)
To: SJ Park; +Cc: Andrew Morton, Asier Gutierrez, damon, linux-kernel, linux-mm
Hi SJ and Asier,
Thanks for working on this and for providing the server results. I have two
questions about what hugepage_mem_bp is intended to represent.
> Introduce DAMOS_QUOTA_HUGEPAGE_MEM_BP auto tuning. Add a new DAMOS quota
> goal metric to measure the amount of huge page consumption to total
> memory consumption ratio.
First, NR_ANON_THPS tracks anonymous PMD mappings, while NR_SHMEM_THPS and
NR_FILE_THPS track PMD-mappable page-cache folios. Therefore, splitting only
an anonymous PMD mapping can change hugepage_mem_bp without physically
splitting the folio. Is the intended metric PMD-mapped memory or physical
large-folio memory? Clarifying this and adding a mapping-only test may help.
Second, the metric is global while a DAMOS scheme can target one process.
THPs from other processes or NUMA nodes can satisfy the target or dilute the
monitored process's changes. Is this intentional? If so, documenting the
scope and testing a background THP workload may be useful.
The temporal results approach the 10% and 25% targets, while the consistent
results overshoot the 10% target to about 20% and 45%. I would describe this
as control-response data. TPS, latency, TLB, fragmentation and collapse CPU
data could further show the workload benefit and cost.
If I have misunderstood any of this, please feel free to ignore these
comments and correct me.
The enum, quota-goal wiring and sysfs exposure otherwise look consistent with
the existing DAMOS autotuning framework. I consider the points above
follow-up questions about semantics and evaluation, rather than blockers for
this series.
For the series:
Reviewed-by: Lian Wang <lianux.mm@gmail.com>
Please feel free to Cc me on related follow-up patches. I am happy to help
review the code. Besides my own DAMON work, I have recently been reviewing
and learning from other DAMON and MM work, and I would be glad to continue.
Thanks,
Lian
Sent using hkml (https://github.com/sjp38/hackermail)
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v4 1/3] mm/damon: Introduce DAMOS_QUOTA_HUGEPAGE auto tuning
2026-09-01 6:58 ` Lian Wang
@ 2026-09-01 14:23 ` SJ Park
2026-09-02 1:56 ` Lian Wang
2026-09-02 15:03 ` Gutierrez Asier
0 siblings, 2 replies; 17+ messages in thread
From: SJ Park @ 2026-09-01 14:23 UTC (permalink / raw)
To: Lian Wang
Cc: SJ Park, Andrew Morton, Asier Gutierrez, damon, linux-kernel,
linux-mm
Hi Lian,
On Tue, 1 Sep 2026 14:58:35 +0800 Lian Wang <lianux.mm@gmail.com> wrote:
> Hi SJ and Asier,
>
> Thanks for working on this and for providing the server results. I have two
> questions about what hugepage_mem_bp is intended to represent.
>
> > Introduce DAMOS_QUOTA_HUGEPAGE_MEM_BP auto tuning. Add a new DAMOS quota
> > goal metric to measure the amount of huge page consumption to total
> > memory consumption ratio.
>
> First, NR_ANON_THPS tracks anonymous PMD mappings, while NR_SHMEM_THPS and
> NR_FILE_THPS track PMD-mappable page-cache folios. Therefore, splitting only
> an anonymous PMD mapping can change hugepage_mem_bp without physically
> splitting the folio. Is the intended metric PMD-mapped memory or physical
> large-folio memory? Clarifying this and adding a mapping-only test may help.
Good point. I was just missing this. So, hugepage_mem_bp is not accounting
anon mTHPs, right? Since hugepage_mem_bp is for general hugepages, I think
this is better to be improved. Seems using MTHP_STAT_NR_ANON that is exposed
as 'nr_anon' can be used? Because this is an improvement rather than a fix of
a bug, I think doing this as either a followup or new version of this series
are ok. Asier, what do you think?
>
> Second, the metric is global while a DAMOS scheme can target one process.
Actually it can target multiple processes if those are in single DAMON context.
> THPs from other processes or NUMA nodes can satisfy the target or dilute the
> monitored process's changes. Is this intentional?
I think it is intentional. The user should have the control on collapsing
hugepages, or believe the uncontrolled collapse mechanisms.
> If so, documenting the
> scope and testing a background THP workload may be useful.
More documentation and testing are always welcome :)
>
> The temporal results approach the 10% and 25% targets, while the consistent
> results overshoot the 10% target to about 20% and 45%. I would describe this
> as control-response data. TPS, latency, TLB, fragmentation and collapse CPU
> data could further show the workload benefit and cost.
Yes, those would be helpful. That's not mandatory for this simple change in my
opinion, though. I would let Asier decide whether and when to make and share
such data.
>
> If I have misunderstood any of this, please feel free to ignore these
> comments and correct me.
Your comments are very helpful, thank you for your review and comments, Lian.
>
> The enum, quota-goal wiring and sysfs exposure otherwise look consistent with
> the existing DAMOS autotuning framework. I consider the points above
> follow-up questions about semantics and evaluation, rather than blockers for
> this series.
I agree.
>
> For the series:
>
> Reviewed-by: Lian Wang <lianux.mm@gmail.com>
Thank you!
>
> Please feel free to Cc me on related follow-up patches. I am happy to help
> review the code. Besides my own DAMON work, I have recently been reviewing
> and learning from other DAMON and MM work, and I would be glad to continue.
I'm curious if you and general reviewers need or prefer to directly be Cc-ed.
I was naively thinking people can search and review DAMON patches by
subscribing to the mailing list or using the archives via lore.kernel.org like
tools.
Thanks,
SJ
[...]
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v4 1/3] mm/damon: Introduce DAMOS_QUOTA_HUGEPAGE auto tuning
2026-09-01 14:23 ` SJ Park
@ 2026-09-02 1:56 ` Lian Wang
2026-09-02 4:23 ` SJ Park
2026-09-02 15:03 ` Gutierrez Asier
1 sibling, 1 reply; 17+ messages in thread
From: Lian Wang @ 2026-09-02 1:56 UTC (permalink / raw)
To: SJ Park; +Cc: Andrew Morton, Asier Gutierrez, damon, linux-kernel, linux-mm
Hi SJ,
> Seems using MTHP_STAT_NR_ANON that is exposed as 'nr_anon' can be used?
Thank you for the suggestion. I will study MTHP_STAT_NR_ANON and nr_anon
further, and document what I find. We can continue discussing it by email,
and I will be happy to review any follow-up or updated patches.
> I'm curious if you and general reviewers need or prefer to directly be Cc-ed.
> I was naively thinking people can search and review DAMON patches by
> subscribing to the mailing list or using the archives via lore.kernel.org like
> tools.
I subscribe to many mailing lists, including linux-mm and DAMON, and I also
follow patches through lore. Direct Cc is not required, but I appreciate it
for DAMON-related patches. Because I care deeply about DAMON, directly Cc-ed
messages are filtered into my important mailbox, so I can archive and review
them quickly.
I am very happy to contribute to DAMON, and I look forward to continuing our
discussions and reviews by email.
Thanks,
Lian
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v4 1/3] mm/damon: Introduce DAMOS_QUOTA_HUGEPAGE auto tuning
2026-09-02 1:56 ` Lian Wang
@ 2026-09-02 4:23 ` SJ Park
2026-09-02 5:12 ` wang lian
0 siblings, 1 reply; 17+ messages in thread
From: SJ Park @ 2026-09-02 4:23 UTC (permalink / raw)
To: Lian Wang
Cc: SJ Park, Andrew Morton, Asier Gutierrez, damon, linux-kernel,
linux-mm
On Wed, 2 Sep 2026 09:56:01 +0800 Lian Wang <lianux.mm@gmail.com> wrote:
> Hi SJ,
>
> > Seems using MTHP_STAT_NR_ANON that is exposed as 'nr_anon' can be used?
>
> Thank you for the suggestion. I will study MTHP_STAT_NR_ANON and nr_anon
> further, and document what I find. We can continue discussing it by email,
> and I will be happy to review any follow-up or updated patches.
>
> > I'm curious if you and general reviewers need or prefer to directly be Cc-ed.
> > I was naively thinking people can search and review DAMON patches by
> > subscribing to the mailing list or using the archives via lore.kernel.org like
> > tools.
>
> I subscribe to many mailing lists, including linux-mm and DAMON, and I also
> follow patches through lore. Direct Cc is not required, but I appreciate it
> for DAMON-related patches. Because I care deeply about DAMON, directly Cc-ed
> messages are filtered into my important mailbox, so I can archive and review
> them quickly.
Thank you so much for your interest in DAMON, Lian. But, the problem is that
my memory is so bad. :'(
I am maintaing mm patches review status dashboard for people who want to find
patches that waiting for their review. It also provides the review status for
each sub-mm components [2] including DAMON [3]. Maybe finding patches that
waiting for your review from the dashboard is also a way for you?
If having the mail directly in your important mailbox is what you need, I
believe adding mails filtering rule for DAMON patches wouldn't be that
difficult. If you use hkml, it could also help you monitoring [4] those.
Yet another systematic and long term solution that I could suggest is keep
reviewing and posting DAMON patches one by one, build more tracked records and
trusts, eventually adding your name as DAMON reviewer or maintainer on the
MAINTAINERS file. All patches sent with get_maintainers.pl will automatically
Cc you. It would take time and require patience, though, of course.
>
> I am very happy to contribute to DAMON, and I look forward to continuing our
> discussions and reviews by email.
I really appreciate your interest and contributions to DAMON. I hope it to be
continued long term!
[1] https://github.com/sjp38/mm_git_dashboard
[2] https://github.com/sjp38/mm_git_dashboard/tree/master/summary/subsystem
[3] https://github.com/sjp38/mm_git_dashboard/tree/master/summary/subsystem/DAMON
[4] https://github.com/sjp38/hackermail/blob/master/USAGE.md#monitoring-mails
Thanks,
SJ
[...]
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v4 1/3] mm/damon: Introduce DAMOS_QUOTA_HUGEPAGE auto tuning
2026-09-02 4:23 ` SJ Park
@ 2026-09-02 5:12 ` wang lian
0 siblings, 0 replies; 17+ messages in thread
From: wang lian @ 2026-09-02 5:12 UTC (permalink / raw)
To: SJ Park; +Cc: Andrew Morton, Asier Gutierrez, damon, linux-kernel, linux-mm
[-- Attachment #1: Type: text/plain, Size: 2922 bytes --]
> On Sep 2, 2026, at 12:23, SJ Park <sj@kernel.org> wrote:
>
> On Wed, 2 Sep 2026 09:56:01 +0800 Lian Wang <lianux.mm@gmail.com <mailto:lianux.mm@gmail.com>> wrote:
>
>> Hi SJ,
>>
>>> Seems using MTHP_STAT_NR_ANON that is exposed as 'nr_anon' can be used?
>>
>> Thank you for the suggestion. I will study MTHP_STAT_NR_ANON and nr_anon
>> further, and document what I find. We can continue discussing it by email,
>> and I will be happy to review any follow-up or updated patches.
>>
>>> I'm curious if you and general reviewers need or prefer to directly be Cc-ed.
>>> I was naively thinking people can search and review DAMON patches by
>>> subscribing to the mailing list or using the archives via lore.kernel.org like
>>> tools.
>>
>> I subscribe to many mailing lists, including linux-mm and DAMON, and I also
>> follow patches through lore. Direct Cc is not required, but I appreciate it
>> for DAMON-related patches. Because I care deeply about DAMON, directly Cc-ed
>> messages are filtered into my important mailbox, so I can archive and review
>> them quickly.
>
> Thank you so much for your interest in DAMON, Lian. But, the problem is that
> my memory is so bad. :'(
>
> I am maintaing mm patches review status dashboard for people who want to find
> patches that waiting for their review. It also provides the review status for
> each sub-mm components [2] including DAMON [3]. Maybe finding patches that
> waiting for your review from the dashboard is also a way for you?
>
> If having the mail directly in your important mailbox is what you need, I
> believe adding mails filtering rule for DAMON patches wouldn't be that
> difficult. If you use hkml, it could also help you monitoring [4] those.
>
> Yet another systematic and long term solution that I could suggest is keep
> reviewing and posting DAMON patches one by one, build more tracked records and
> trusts, eventually adding your name as DAMON reviewer or maintainer on the
> MAINTAINERS file. All patches sent with get_maintainers.pl will automatically
> Cc you. It would take time and require patience, though, of course.
Thank you for your detailed feedback.
>
>>
>> I am very happy to contribute to DAMON, and I look forward to continuing our
>> discussions and reviews by email.
>
> I really appreciate your interest and contributions to DAMON. I hope it to be
> continued long term!
>
> [1] https://github.com/sjp38/mm_git_dashboard
> [2] https://github.com/sjp38/mm_git_dashboard/tree/master/summary/subsystem
> [3] https://github.com/sjp38/mm_git_dashboard/tree/master/summary/subsystem/DAMON
> [4] https://github.com/sjp38/hackermail/blob/master/USAGE.md#monitoring-mails
>
Okay, I will. I will contribute to the community, continuously learn and provide feedback to
ultimately make Damon better.
>
>
> Thanks,
> SJ
>
Thanks
Lian
> [...]
[-- Attachment #2: Type: text/html, Size: 30259 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v4 1/3] mm/damon: Introduce DAMOS_QUOTA_HUGEPAGE auto tuning
2026-09-01 14:23 ` SJ Park
2026-09-02 1:56 ` Lian Wang
@ 2026-09-02 15:03 ` Gutierrez Asier
2026-09-02 15:15 ` SJ Park
1 sibling, 1 reply; 17+ messages in thread
From: Gutierrez Asier @ 2026-09-02 15:03 UTC (permalink / raw)
To: SJ Park, Lian Wang; +Cc: Andrew Morton, damon, linux-kernel, linux-mm
Hi SJ and Lian,
On 9/1/2026 5:23 PM, SJ Park wrote:
> Hi Lian,
>
> On Tue, 1 Sep 2026 14:58:35 +0800 Lian Wang <lianux.mm@gmail.com> wrote:
>
>> Hi SJ and Asier,
>>
>> Thanks for working on this and for providing the server results. I have two
>> questions about what hugepage_mem_bp is intended to represent.
>>
>>> Introduce DAMOS_QUOTA_HUGEPAGE_MEM_BP auto tuning. Add a new DAMOS quota
>>> goal metric to measure the amount of huge page consumption to total
>>> memory consumption ratio.
>>
>> First, NR_ANON_THPS tracks anonymous PMD mappings, while NR_SHMEM_THPS and
>> NR_FILE_THPS track PMD-mappable page-cache folios. Therefore, splitting only
>> an anonymous PMD mapping can change hugepage_mem_bp without physically
>> splitting the folio. Is the intended metric PMD-mapped memory or physical
>> large-folio memory? Clarifying this and adding a mapping-only test may help.
>
> Good point. I was just missing this. So, hugepage_mem_bp is not accounting
> anon mTHPs, right? Since hugepage_mem_bp is for general hugepages, I think
> this is better to be improved. Seems using MTHP_STAT_NR_ANON that is exposed
> as 'nr_anon' can be used? Because this is an improvement rather than a fix of
> a bug, I think doing this as either a followup or new version of this series
> are ok. Asier, what do you think?
Agree, I will submit a new patch to fix this behaviour and account for mTHP.
Lian, thanks a lot for the review and pointing this issue.>>
>> Second, the metric is global while a DAMOS scheme can target one process.
>
> Actually it can target multiple processes if those are in single DAMON context.
>
>> THPs from other processes or NUMA nodes can satisfy the target or dilute the
>> monitored process's changes. Is this intentional?
>
> I think it is intentional. The user should have the control on collapsing
> hugepages, or believe the uncontrolled collapse mechanisms.
>
>> If so, documenting the
>> scope and testing a background THP workload may be useful.
>
> More documentation and testing are always welcome :)
>
>>
>> The temporal results approach the 10% and 25% targets, while the consistent
>> results overshoot the 10% target to about 20% and 45%. I would describe this
>> as control-response data. TPS, latency, TLB, fragmentation and collapse CPU
>> data could further show the workload benefit and cost.
>
> Yes, those would be helpful. That's not mandatory for this simple change in my
> opinion, though. I would let Asier decide whether and when to make and share
> such data.
The idea of this patch was to introduce auto tuning. I think I submitted thebenchmark results when I sent the DAMOS_COLLAPSE feature.
Anyway, it shouldn't take long for me to get more data. I think it may be useful
for people to know the actual results in real application.
SJ, what's the best place to publish all those data? I believe you have more
experience sharing this data and which platform is the most useful one.>>
>> If I have misunderstood any of this, please feel free to ignore these
>> comments and correct me.
>
> Your comments are very helpful, thank you for your review and comments, Lian.
>
>>
>> The enum, quota-goal wiring and sysfs exposure otherwise look consistent with
>> the existing DAMOS autotuning framework. I consider the points above
>> follow-up questions about semantics and evaluation, rather than blockers for
>> this series.
>
> I agree.
>
>>
>> For the series:
>>
>> Reviewed-by: Lian Wang <lianux.mm@gmail.com>
>
> Thank you!
>
>>
>> Please feel free to Cc me on related follow-up patches. I am happy to help
>> review the code. Besides my own DAMON work, I have recently been reviewing
>> and learning from other DAMON and MM work, and I would be glad to continue.
>
> I'm curious if you and general reviewers need or prefer to directly be Cc-ed.
> I was naively thinking people can search and review DAMON patches by
> subscribing to the mailing list or using the archives via lore.kernel.org like
> tools.
>
>
> Thanks,
> SJ
>
> [...]
--
Asier Gutierrez
Huawei
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v4 1/3] mm/damon: Introduce DAMOS_QUOTA_HUGEPAGE auto tuning
2026-09-02 15:03 ` Gutierrez Asier
@ 2026-09-02 15:15 ` SJ Park
0 siblings, 0 replies; 17+ messages in thread
From: SJ Park @ 2026-09-02 15:15 UTC (permalink / raw)
To: Gutierrez Asier
Cc: SJ Park, Lian Wang, Andrew Morton, damon, linux-kernel, linux-mm
On Wed, 2 Sep 2026 18:03:39 +0300 Gutierrez Asier <gutierrez.asier@huawei-partners.com> wrote:
> Hi SJ and Lian,
>
> On 9/1/2026 5:23 PM, SJ Park wrote:
> > Hi Lian,
> >
> > On Tue, 1 Sep 2026 14:58:35 +0800 Lian Wang <lianux.mm@gmail.com> wrote:
> >
[...]
> >> Second, the metric is global while a DAMOS scheme can target one process.
> >
> > Actually it can target multiple processes if those are in single DAMON context.
> >
> >> THPs from other processes or NUMA nodes can satisfy the target or dilute the
> >> monitored process's changes. Is this intentional?
> >
> > I think it is intentional. The user should have the control on collapsing
> > hugepages, or believe the uncontrolled collapse mechanisms.
> >
> >> If so, documenting the
> >> scope and testing a background THP workload may be useful.
> >
> > More documentation and testing are always welcome :)
> >
> >>
> >> The temporal results approach the 10% and 25% targets, while the consistent
> >> results overshoot the 10% target to about 20% and 45%. I would describe this
> >> as control-response data. TPS, latency, TLB, fragmentation and collapse CPU
> >> data could further show the workload benefit and cost.
> >
> > Yes, those would be helpful. That's not mandatory for this simple change in my
> > opinion, though. I would let Asier decide whether and when to make and share
> > such data.
> The idea of this patch was to introduce auto tuning. I think I submitted thebenchmark results when I sent the DAMOS_COLLAPSE feature.
>
> Anyway, it shouldn't take long for me to get more data. I think it may be useful
> for people to know the actual results in real application.
>
> SJ, what's the best place to publish all those data? I believe you have more
> experience sharing this data and which platform is the most useful one.>>
I'm never an expert in visibility :) But if I should recommend, I get three
places off the top of my head.
First, you could simply share the data with plain mail on DAMON and wider
kernel mailing lists like linux-mm@ and linux-kernel@.
Second, if the data is matured, sharing those in conferences like LPC,
OSSummit, FOSDEM or DAMON (nano) Conference would be nice.
Third, if you prefer text, LWN or personal blogs like medias could also be good
places. I also maintain DAMON porject blog [1]. Kunwu and Lian actually
contributed an article to the blog before. If you want to submit an article
there, let me know.
[1] https://damonitor.github.io/posts/
[2] https://damonitor.github.io/posts/tlb_flush_policy/
Thanks,
SJ
[...]
^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2026-09-02 15:15 UTC | newest]
Thread overview: 17+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-08-31 14:47 [PATCH v4 0/3] mm/damon: Introduce a huge page collapsing mechanism using auto tuning SJ Park
2026-08-31 14:47 ` [PATCH v4 1/3] mm/damon: Introduce DAMOS_QUOTA_HUGEPAGE " SJ Park
2026-09-01 6:58 ` Lian Wang
2026-09-01 14:23 ` SJ Park
2026-09-02 1:56 ` Lian Wang
2026-09-02 4:23 ` SJ Park
2026-09-02 5:12 ` wang lian
2026-09-02 15:03 ` Gutierrez Asier
2026-09-02 15:15 ` SJ Park
2026-08-31 14:47 ` [PATCH v4 2/3] mm/damon/sysfs: support hugepage_mem_bp quota goal metric SJ Park
2026-08-31 14:47 ` [PATCH v4 3/3] Docs/mm/damon/design: Document hugepage_mem_bp target metric SJ Park
2026-08-31 15:45 ` Randy Dunlap
2026-08-31 20:00 ` Randy Dunlap
2026-09-01 0:47 ` SJ Park
2026-09-01 5:20 ` Gutierrez Asier
2026-09-01 5:31 ` SJ Park
2026-09-01 0:49 ` [PATCH v4 0/3] mm/damon: Introduce a huge page collapsing mechanism using auto tuning SJ Park
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox