[PATCH v2 0/3] mm/damon: Introduce a huge page collapsing mechanism using auto tuning

[PATCH v2 0/3] mm/damon: Introduce a huge page collapsing mechanism using auto tuning

[gutierrez.asier]

From: Asier Gutierrez <gutierrez.asier@huawei-partners.com>

Overview
========

This patch set introduces a new autotuning which allows to collapse
hot regions into hugepages.

Motivation
==========

Since TLB is a bottleneck for many systems[1], a way to optimize TLB
misses (or hits) is to use huge pages. Unfortunately, using "always"
in THP leads to memory fragmentation and memory waste. For this reason,
most application guides and system administrators suggest to disable THP.

Currently DAMON has DAMOS_HUGEPAGE, DAMOS_NONHUGEPAGE and DAMOS_COLLAPSE.
However, there is no way to tune the settings. It will collapse all the
hot regions that meet the access pattern. If the server is a bare metal
database or big data server, this will also lead to eventual fragmentation.

Additionally, currently THP is set globally. Ideally, there should be a
way to control which tasks can use huge pages.

Solution
========

DAMON has now a way to autotune some of the variables and adjust quotas
automatically, so that DAMON is fired only under the right circumstances.
It would be nice to have something similar, but for huge pages.

A new autotuning quota goal[2], damos_hugepage_mem_bp, is introduced,
which checks the huge page consumption to total memory consumption. This
new quota mechanism reuses current autotuning architecture.

In order to test this new mechanism, a sample module[3] was created, but
not included in this patch series, which demonstrates the use of huge
pages collapse autotuning.

Benchmarks
==========

Huge page collapse autotuning was tested in a physicial machine with
MariaDB 10.5.29 and sysbench as the benchmark framework.

The hugepage module was set up in the following way:

# echo 1000 > min_age
# echo 1000 > quota_percentage_hugepage
# echo $(pidof mariadbd) > taget_pid
# echo on > enabled

The goal was to achieve 5% of the total memory used as hugepage.
Since the database was not very big, we may not be able to achieve
high amount of huge pages per total memory consumption ratio.

The table below shows the memory consumption over time. Timestamp is in
second and the memory usage in is MBytes. Gaps in the timestamp means
that no changes in the hugepage consumption happened over that period
of time in MB. The total used memory is calculated as
mem_total - mem free. The huge page used is calculated as
huge_page_anon + huge_page_shmem + huge_page_file. The table also
shows the huge pages to total memory ratio.

Hugepage autotune benchmark:
+-----------+----------------+----------------+----------------------+
| timestamp | total mem used | huge page used | percentage hugepage  |
+-----------+----------------+----------------+----------------------+
| 0         | 3044.988281    | 0              | 0%                   |
| 22        | 3160.207031    | 2              | 0.06%                |
| 30        | 3250.90625     | 4              | 0.12%                |
| 69        | 3781.238281    | 6              | 0.16%                |
| 71        | 3822.226563    | 8              | 0.21%                |
| 72        | 3846.578125    | 10             | 0.26%                |
| 73        | 3852.402344    | 12             | 0.31%                |
| 74        | 3868           | 14             | 0.36%                |
| 75        | 3881.84375     | 104            | 2.68%                |
| 275       | 4194.175781    | 106            | 2.52%                |
+-----------+----------------+----------------+----------------------+
After second 275, no more pages are collapsed into hugepages


THP (always) benchmark:
+-----------+----------------+----------------+---------------------+
| timestamp | total mem used | huge page used | percentage hugepage |
+-----------+----------------+----------------+---------------------+
|         1 |    4489.320313 |            184 |         4.098615986 |
|        15 |    4581.871094 |            214 |         4.670580984 |
|        30 |    4757.742188 |            376 |         7.902908253 |
|        45 |    4937.574219 |            558 |         11.30109595 |
|        60 |    5147.867188 |            728 |         14.14177898 |
|        75 |      5407.0625 |            918 |         16.97779524 |
|        95 |    5668.796875 |           1040 |         18.34604455 |
|       105 |    5723.839844 |           1056 |         18.44915352 |
|       115 |     5736.84375 |           1072 |         18.68623317 |
|       125 |    5732.042969 |           1088 |         18.98101612 |
|       186 |    5753.601563 |           1184 |         20.57841488 |
|       246 |    5746.398438 |           1280 |         22.27482159 |
|       306 |    5752.128906 |           1376 |         23.92157795 |
|       367 |      5772.5625 |           1472 |         25.49994045 |
|       427 |    5832.019531 |           1568 |         26.88605536 |
|       488 |    5813.246094 |           1664 |         28.62428277 |
|       548 |    5807.621094 |           1760 |         30.30500736 |
|       598 |    5841.253906 |           1822 |         31.19193292 |
|       669 |    5982.160156 |           1854 |         30.99214918 |
|       931 |    5946.605469 |           1868 |         31.41287933 |
|       981 |    6020.207031 |           1896 |         31.49393352 |
|       991 |    5988.445313 |           1910 |         31.89475566 |
|      1011 |    5988.570313 |           1926 |         32.16126554 |
|      1032 |    6016.039063 |           1936 |         32.18064211 |
|      1575 |    6057.289063 |           1968 |         32.48978181 |
|      1606 |    6026.167969 |           2000 |         33.18858702 |
+-----------+----------------+----------------+---------------------+
I ignored some points to make the table shorter. Anyway, the amount
of memory consumption, total and huge pages, is a lot higher than
with DAMON hugepage autotuning.


Performance:
Baseline (no THP, module off) -> 18,162.45 transactions per second
Hugepage autotune -> 18,211.82 transactions per second (+0.27% improvement)
THP always -> 18,388.3 (+1.24%)
THP madvise -> 18,179.25 (+0.09%)

Improvement is due to lower TLB misses

Patches Sequence
================
Patch 1 -> Introduce DAMOS_QUOTA_HUGEPAGE_MEM_BP and autotuning
Patch 2 -> Module that demonstrates how to use
           DAMOS_QUOTA_HUGEPAGE_MEM_BP and DAMOS_QUOTA_GOAL_TUNER_TEMPORAL
Patch 3 -> Document hugepage_mem_bp parameter

Changes from previous versions
==============================
v1[4] -> v2
  - Removed the sample module altogether, since it will
    increase maintenance costs[5]
  - Document hugepage_mem_bp in design.rst

RFC 4[6] -> v1
  - Renamed config to SAMPLE_DAMON_HPAGE, file to hpage.c and
    functions to damon_sample_hpage_...
  - Make the module depend on TRANSPARENT_HUGEPAGE, since
    the module will need some THP functions anyway
  - Removed documentation, since this is just a sample module
  - Removed DAMOS_QUOTA_HUGEPAGE_MEM_BP from
    damos_sysfs_add_quota_score
  - Added a short description of the module in Kconfig

RFC 3[7] -> RFC 4
  - Simplified the module
  - Removed unnecessary parameters
  - Renamed DAMOS_QUOTA_HUGEPAGE_MEM_BP to unify the
    naming style
  - Switched to DAMOS_QUOTA_GOAL_TUNER_TEMPORAL
  - Updated the documentation
  - Removed new interface for context creation with
    DAMON_OPS_VADDR

RFC 2[8] -> RFC 3
  - Module moved to samples
  - Change autotune to monitor total memory and hugepage
  - Added performnace benchmarks to the cover letter
  - Bail out gracefully when trying to start disable
    the module after the monitored task exited. This
    issue was discovered by sashiko [9]
  - Fixed typos and added quota_sz to the documentation
    discovered by sashiko [10]

RFC 1[11] -> RFC 2
  - Rebased into mm-new
  - Use DAMOS_COLLAPSE instead of DAMOS_HUGEPAGE
  - Fixed an issue that returned silently an error when the PID
    didn't exist in the system.[12]

[1] https://dl.acm.org/doi/pdf/10.1145/3307650.3322227
[2] https://lore.kernel.org/e67f05ad-dbb9-45e6-ba30-b167a99ac67d@huawei-partners.com
[3] https://lore.kernel.org/20260616150316.580819-3-gutierrez.asier@huawei-partners.com
[4] https://lore.kernel.org/20260616150316.580819-1-gutierrez.asier@huawei-partners.com
[5] https://lore.kernel.org/20260618150806.4633-1-sj@kernel.org
[6] https://lore.kernel.org/20260611150244.3454699-1-gutierrez.asier@huawei-partners.com
[7] https://lore.kernel.org/20260604150338.501128-1-gutierrez.asier@huawei-partners.com
[8] https://lore.kernel.org/20260522145518.158910-1-gutierrez.asier@huawei-partners.com
[9] https://lore.kernel.org/20260522171210.900B11F00A3D@smtp.kernel.org
[10] https://lore.kernel.org/20260522171633.AAF5B1F000E9@smtp.kernel.org
[11] https://lore.kernel.org/20260430134139.2446417-1-gutierrez.asier@huawei-partners.com
[12] https://lore.kernel.org/all/20260430154338.E22E6C2BCB3@smtp.kernel.org/

Asier Gutierrez (3):
  mm/damon: Introduce DAMOS_QUOTA_HUGEPAGE auto tuning
  mm/damon/sysfs: support hugepage_mem_bp quota goal metric
  Docs/mm/damon/design: Document hugepage_mem_bp target metric

 Documentation/mm/damon/design.rst |  2 ++
 include/linux/damon.h             |  2 ++
 mm/damon/core.c                   | 14 ++++++++++++++
 mm/damon/sysfs-schemes.c          |  4 ++++
 4 files changed, 22 insertions(+)

-- 
2.43.0

[PATCH v2 1/3] mm/damon: Introduce DAMOS_QUOTA_HUGEPAGE auto tuning

[gutierrez.asier]

From: Asier Gutierrez <gutierrez.asier@huawei-partners.com>

Introduce DAMOS_QUOTA_HUGEPAGE_MEM_BP auto tuning. Add a new
DAMOS quota goal metric to measure the amount of huge page
consumption to total memory consumption ratio.

Signed-off-by: Asier Gutierrez <gutierrez.asier@huawei-partners.com>
---
 include/linux/damon.h |  2 ++
 mm/damon/core.c       | 14 ++++++++++++++
 2 files changed, 16 insertions(+)

diff --git a/include/linux/damon.h b/include/linux/damon.h
index 6f7edb3590ef..23a9cec05033 100644
--- a/include/linux/damon.h
+++ b/include/linux/damon.h
@@ -162,6 +162,7 @@ enum damos_action {
  * @DAMOS_QUOTA_INACTIVE_MEM_BP:	Inactive to total LRU memory ratio.
  * @DAMOS_QUOTA_NODE_ELIGIBLE_MEM_BP:	Scheme-eligible memory ratio of a
  *					node in basis points (0-10000).
+ * @DAMOS_QUOTA_HUGEPAGE_MEM_BP:	Huge page to total used memory ratio.
  * @NR_DAMOS_QUOTA_GOAL_METRICS:	Number of DAMOS quota goal metrics.
  *
  * Metrics equal to larger than @NR_DAMOS_QUOTA_GOAL_METRICS are unsupported.
@@ -176,6 +177,7 @@ enum damos_quota_goal_metric {
 	DAMOS_QUOTA_ACTIVE_MEM_BP,
 	DAMOS_QUOTA_INACTIVE_MEM_BP,
 	DAMOS_QUOTA_NODE_ELIGIBLE_MEM_BP,
+	DAMOS_QUOTA_HUGEPAGE_MEM_BP,
 	NR_DAMOS_QUOTA_GOAL_METRICS,
 };
 
diff --git a/mm/damon/core.c b/mm/damon/core.c
index 7e4b9affc5b0..b001f80681b1 100644
--- a/mm/damon/core.c
+++ b/mm/damon/core.c
@@ -2795,6 +2795,17 @@ static unsigned int damos_get_in_active_mem_bp(bool active_ratio)
 	return mult_frac(inactive, 10000, total);
 }
 
+static unsigned int damos_hugepage_mem_bp(void)
+{
+	unsigned long thp, total;
+
+	thp = global_node_page_state(NR_ANON_THPS) +
+				global_node_page_state(NR_SHMEM_THPS) +
+				global_node_page_state(NR_FILE_THPS);
+	total = totalram_pages() - global_zone_page_state(NR_FREE_PAGES);
+	return mult_frac(thp, 10000, total);
+}
+
 static void damos_set_quota_goal_current_value(struct damon_ctx *c,
 		struct damos *s, struct damos_quota_goal *goal)
 {
@@ -2826,6 +2837,9 @@ static void damos_set_quota_goal_current_value(struct damon_ctx *c,
 		goal->current_value = damos_get_node_eligible_mem_bp(c, s,
 				goal->nid);
 		break;
+	case DAMOS_QUOTA_HUGEPAGE_MEM_BP:
+		goal->current_value = damos_hugepage_mem_bp();
+		break;
 	default:
 		break;
 	}
-- 
2.43.0

[PATCH v2 2/3] mm/damon/sysfs: support hugepage_mem_bp quota goal metric

[gutierrez.asier]

From: Asier Gutierrez <gutierrez.asier@huawei-partners.com>

DAMOS has a new autotune policy metric: DAMOS_QUOTA_HUGEPAGE_MEM_BP.
This patch exposes DAMOS_QUOTA_HUGEPAGE_MEM_BP through sysfs.

Add the "hugepage_mem_bp" to the sysfs-schemes interface.

Signed-off-by: Asier Gutierrez <gutierrez.asier@huawei-partners.com>
Reviewed-by: SeongJae Park <sj@kernel.org>
---
 mm/damon/sysfs-schemes.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/mm/damon/sysfs-schemes.c b/mm/damon/sysfs-schemes.c
index 329cfd0bbe9f..4b5429a5b415 100644
--- a/mm/damon/sysfs-schemes.c
+++ b/mm/damon/sysfs-schemes.c
@@ -1267,6 +1267,10 @@ struct damos_sysfs_qgoal_metric_name damos_sysfs_qgoal_metric_names[] = {
 		.metric = DAMOS_QUOTA_NODE_ELIGIBLE_MEM_BP,
 		.name = "node_eligible_mem_bp",
 	},
+	{
+		.metric = DAMOS_QUOTA_HUGEPAGE_MEM_BP,
+		.name = "hugepage_mem_bp",
+	},
 };
 
 static ssize_t target_metric_show(struct kobject *kobj,
-- 
2.43.0

[PATCH v2 3/3] Docs/mm/damon/design: Document hugepage_mem_bp target metric

[gutierrez.asier]

From: Asier Gutierrez <gutierrez.asier@huawei-partners.com>

Document huge_mem_bp metric exposed by sysfs.

Signed-off-by: Asier Gutierrez <gutierrez.asier@huawei-partners.com>
---
 Documentation/mm/damon/design.rst | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/Documentation/mm/damon/design.rst b/Documentation/mm/damon/design.rst
index 2da7ca0d3d17..34d940a13a85 100644
--- a/Documentation/mm/damon/design.rst
+++ b/Documentation/mm/damon/design.rst
@@ -686,6 +686,8 @@ mechanism tries to make ``current_value`` of ``target_metric`` be same to
   (1/10,000).
 - ``inactive_mem_bp``: Inactive to active + inactive (LRU) memory size ratio in
   bp (1/10,000).
+- ``hugepage_mem_bp``: Total huge page to total used memory ratio in bp
+  (1/10,000).
 
 ``nid`` is optionally required for only ``node_mem_used_bp``,
 ``node_mem_free_bp``, ``node_memcg_used_bp`` and ``node_memcg_free_bp`` to
-- 
2.43.0

Re: [PATCH v2 0/3] mm/damon: Introduce a huge page collapsing mechanism using auto tuning

[SeongJae Park]

Hi Asier,


Thank you for keep working on this.

But, I don't think you answered all my questions to the previous revision.
Correct me if I'm wrong.  If I'm not incorrect, could you please make sure all
comments are addressed before sending a new revision, from next time?  I'm
holding reviewing this revision until all my comments on the previous reevision
are addressed.


Thanks,
SJ

[...]