[PATCH 00/11] mm/damon: support multiple goal-based quota tuning algorithms

[PATCH 00/11] mm/damon: support multiple goal-based quota tuning algorithms

[SeongJae Park]

Aim-oriented DAMOS quota auto-tuning uses a single tuning algorithm.
The algorithm is designed to find a quota value that should be
consistently kept for achieving the aimed goal for long term.  It is
useful and reliable at automatically operating systems that have dynamic
environments in the long term.

As always, however, no single algorithm fits all.  When the environment
has static characteristics or there are control towers in not only the
kernel space but also the user space, the algorithm shows some
limitations.  In such environments, users want kernel work in a more
short term deterministic way.  Actually there were at least two reports
[1,2] of such cases.

Extend DAMOS quotas goal to support multiple quota tuning algorithms
that users can select.  Keep the current algorithm as the default one,
to not break the old users.  Also give it a name, "consist", as it is
designed to "consistently" apply the DAMOS action.  And introduce a new
tuning algorithm, namely "temporal".  It is designed to apply the DAMOS
action only temporally, in a deterministic way.  In more detail, as long
as the goal is under-achieved, it uses the maximum quota available.
Once the goal is over-achieved, it sets the quota zero.

Tests
=====

I confirmed the feature is working as expected using the latest version
of DAMON user-space tool, like below.

    $ # start DAMOS for reclaiming memory aiming 30% free memory
    $ sudo ./damo/damo start --damos_action pageout \
            --damos_quota_goal_tuner temporal \
            --damos_quota_goal node_mem_free_bp 30% 0 \
            --damos_quota_interval 1s \
            --damos_quota_space 100M

Note that >=3.1.8 version of DAMON user-space tool supports this feature
(--damos_quota_goal_tuner).  As expected, DAMOS stops reclaiming memory
as soon as the goal amount of free memory is made.  When 'consist' tuner
is used, the reclamation was continued even after the goal amount of
free memory is made, resulting in more than goal amount of free memory,
as expected.

Patch Sequence
==============

First four patches implement the features.  Patch 1 extends core API to
allow multiple tuners and make the current tuner as the default and only
available tuner, namely 'consist'.  Patch 2 allows future tuners setting
zero effective quota.  Patch 3 introduces the second tuner, namely
'temporal'.  Patch 4 further extends DAMON sysfs API to let users use
that.

Three following patches (patches 5-7) update design, usage, and ABI
documents, respectively.

Final four patches (patches 8-11) are for adding tests.  The eighth
patch (patch 8) extends the kunit test for online parameters commit for
validating the goal_tuner.  The ninth and the tenth patches (patches
9-10) extend the testing-purpose DAMON sysfs control helper and DAMON
status dumping tool to support the newly added feature.  The final
eleventh one (patch 11) extends the existing online commit selftest to
cover the new feature.

References
==========

[1] https://lore.kernel.org/CALa+Y17__d=ZsM1yX+MXx0ozVdsXnFqF4p0g+kATEitrWyZFfg@mail.gmail.com
[2] https://lore.kernel.org/20260204022537.814-1-yunjeong.mun@sk.com

Changelog
=========

Changes from RFC v2
(https://lore.kernel.org/20260304044122.79394-1-sj@kernel.org)
- Rename damos_quota_set() to damos_quota_is_set().
- Drop unnecessary ->goal_tuner set on damos_new_quota().
- Add kunit test.
- Add goal_tuenr kernel-doc comment.

Changes from RFC v1
(https://lore.kernel.org/20260212062314.69961-1-sj@kernel.org)
- Add selftest for goal_tuner commitment.
- Set goal tuner inside damon_new_scheme().
- Allow zero size effective size quota.
- Update the ABI document.
- Wordsmith change descriptions.

SeongJae Park (11):
  mm/damon/core: introduce damos_quota_goal_tuner
  mm/damon/core: allow quota goals set zero effective size quota
  mm/damon/core: introduce DAMOS_QUOTA_GOAL_TUNER_TEMPORAL
  mm/damon/sysfs-schemes: implement quotas->goal_tuner file
  Docs/mm/damon/design: document the goal-based quota tuner selections
  Docs/admin-guide/mm/damon/usage: document goal_tuner sysfs file
  Docs/ABI/damon: update for goal_tuner
  mm/damon/tests/core-kunit: test goal_tuner commit
  selftests/damon/_damon_sysfs: support goal_tuner setup
  selftests/damon/drgn_dump_damon_status: support quota goal_tuner
    dumping
  selftests/damon/sysfs.py: test goal_tuner commit

 .../ABI/testing/sysfs-kernel-mm-damon         |  6 ++
 Documentation/admin-guide/mm/damon/usage.rst  | 16 +++--
 Documentation/mm/damon/design.rst             | 12 ++++
 include/linux/damon.h                         | 12 ++++
 mm/damon/core.c                               | 61 ++++++++++++++++---
 mm/damon/sysfs-schemes.c                      | 58 ++++++++++++++++++
 mm/damon/tests/core-kunit.h                   |  3 +
 tools/testing/selftests/damon/_damon_sysfs.py | 12 +++-
 .../selftests/damon/drgn_dump_damon_status.py |  1 +
 tools/testing/selftests/damon/sysfs.py        |  7 +++
 10 files changed, 171 insertions(+), 17 deletions(-)


base-commit: 65dad07b05cbdda486d599772f908f1df45dd306
-- 
2.47.3

[PATCH 08/11] mm/damon/tests/core-kunit: test goal_tuner commit

[SeongJae Park]

Extend damos_commit_quota() kunit test for the newly added goal_tuner
parameter.

Signed-off-by: SeongJae Park <sj@kernel.org>
---
 mm/damon/tests/core-kunit.h | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/mm/damon/tests/core-kunit.h b/mm/damon/tests/core-kunit.h
index e86d4f4fe261a..9e5904c2beeb2 100644
--- a/mm/damon/tests/core-kunit.h
+++ b/mm/damon/tests/core-kunit.h
@@ -693,6 +693,7 @@ static void damos_test_commit_quota(struct kunit *test)
 		.reset_interval = 1,
 		.ms = 2,
 		.sz = 3,
+		.goal_tuner = DAMOS_QUOTA_GOAL_TUNER_CONSIST,
 		.weight_sz = 4,
 		.weight_nr_accesses = 5,
 		.weight_age = 6,
@@ -701,6 +702,7 @@ static void damos_test_commit_quota(struct kunit *test)
 		.reset_interval = 7,
 		.ms = 8,
 		.sz = 9,
+		.goal_tuner = DAMOS_QUOTA_GOAL_TUNER_TEMPORAL,
 		.weight_sz = 10,
 		.weight_nr_accesses = 11,
 		.weight_age = 12,
@@ -714,6 +716,7 @@ static void damos_test_commit_quota(struct kunit *test)
 	KUNIT_EXPECT_EQ(test, dst.reset_interval, src.reset_interval);
 	KUNIT_EXPECT_EQ(test, dst.ms, src.ms);
 	KUNIT_EXPECT_EQ(test, dst.sz, src.sz);
+	KUNIT_EXPECT_EQ(test, dst.goal_tuner, src.goal_tuner);
 	KUNIT_EXPECT_EQ(test, dst.weight_sz, src.weight_sz);
 	KUNIT_EXPECT_EQ(test, dst.weight_nr_accesses, src.weight_nr_accesses);
 	KUNIT_EXPECT_EQ(test, dst.weight_age, src.weight_age);
-- 
2.47.3

[PATCH 09/11] selftests/damon/_damon_sysfs: support goal_tuner setup

[SeongJae Park]

Add support of goal_tuner setup to the test-purpose DAMON sysfs
interface control helper, _damon_sysfs.py.

Signed-off-by: SeongJae Park <sj@kernel.org>
---
 tools/testing/selftests/damon/_damon_sysfs.py | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/tools/testing/selftests/damon/_damon_sysfs.py b/tools/testing/selftests/damon/_damon_sysfs.py
index 748778b563cd2..2b4df655d9fd0 100644
--- a/tools/testing/selftests/damon/_damon_sysfs.py
+++ b/tools/testing/selftests/damon/_damon_sysfs.py
@@ -130,15 +130,16 @@ class DamosQuota:
     sz = None                   # size quota, in bytes
     ms = None                   # time quota
     goals = None                # quota goals
+    goal_tuner = None           # quota goal tuner
     reset_interval_ms = None    # quota reset interval
     weight_sz_permil = None
     weight_nr_accesses_permil = None
     weight_age_permil = None
     scheme = None               # owner scheme
 
-    def __init__(self, sz=0, ms=0, goals=None, reset_interval_ms=0,
-                 weight_sz_permil=0, weight_nr_accesses_permil=0,
-                 weight_age_permil=0):
+    def __init__(self, sz=0, ms=0, goals=None, goal_tuner='consist',
+                 reset_interval_ms=0, weight_sz_permil=0,
+                 weight_nr_accesses_permil=0, weight_age_permil=0):
         self.sz = sz
         self.ms = ms
         self.reset_interval_ms = reset_interval_ms
@@ -146,6 +147,7 @@ class DamosQuota:
         self.weight_nr_accesses_permil = weight_nr_accesses_permil
         self.weight_age_permil = weight_age_permil
         self.goals = goals if goals is not None else []
+        self.goal_tuner = goal_tuner
         for idx, goal in enumerate(self.goals):
             goal.idx = idx
             goal.quota = self
@@ -191,6 +193,10 @@ class DamosQuota:
             err = goal.stage()
             if err is not None:
                 return err
+        err = write_file(
+                os.path.join(self.sysfs_dir(), 'goal_tuner'), self.goal_tuner)
+        if err is not None:
+            return err
         return None
 
 class DamosWatermarks:
-- 
2.47.3

[PATCH 10/11] selftests/damon/drgn_dump_damon_status: support quota goal_tuner dumping

[SeongJae Park]

Update drgn_dump_damon_status.py, which is being used to dump the
in-kernel DAMON status for tests, to dump goal_tuner setup status.

Signed-off-by: SeongJae Park <sj@kernel.org>
---
 tools/testing/selftests/damon/drgn_dump_damon_status.py | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tools/testing/selftests/damon/drgn_dump_damon_status.py b/tools/testing/selftests/damon/drgn_dump_damon_status.py
index 5374d18d1fa8d..af99b07a4f565 100755
--- a/tools/testing/selftests/damon/drgn_dump_damon_status.py
+++ b/tools/testing/selftests/damon/drgn_dump_damon_status.py
@@ -110,6 +110,7 @@ def damos_quota_to_dict(quota):
         ['reset_interval', int],
         ['ms', int], ['sz', int],
         ['goals', damos_quota_goals_to_list],
+        ['goal_tuner', int],
         ['esz', int],
         ['weight_sz', int],
         ['weight_nr_accesses', int],
-- 
2.47.3

[PATCH 11/11] selftests/damon/sysfs.py: test goal_tuner commit

[SeongJae Park]

Extend the near-full DAMON parameters commit selftest to commit
goal_tuner and confirm the internal status is updated as expected.

Signed-off-by: SeongJae Park <sj@kernel.org>
---
 tools/testing/selftests/damon/sysfs.py | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/tools/testing/selftests/damon/sysfs.py b/tools/testing/selftests/damon/sysfs.py
index 9cca71eb0325e..3aa5c91548a53 100755
--- a/tools/testing/selftests/damon/sysfs.py
+++ b/tools/testing/selftests/damon/sysfs.py
@@ -67,6 +67,12 @@ def assert_quota_committed(quota, dump):
     assert_true(dump['sz'] == quota.sz, 'sz', dump)
     for idx, qgoal in enumerate(quota.goals):
         assert_quota_goal_committed(qgoal, dump['goals'][idx])
+    tuner_val = {
+            'consist': 0,
+            'temporal': 1,
+            }
+    assert_true(dump['goal_tuner'] == tuner_val[quota.goal_tuner],
+                'goal_tuner', dump)
     assert_true(dump['weight_sz'] == quota.weight_sz_permil, 'weight_sz', dump)
     assert_true(dump['weight_nr_accesses'] == quota.weight_nr_accesses_permil,
                 'weight_nr_accesses', dump)
@@ -231,6 +237,7 @@ def main():
                         metric='node_mem_used_bp',
                         target_value=9950,
                         nid=1)],
+                    goal_tuner='temporal',
                     reset_interval_ms=1500,
                     weight_sz_permil=20,
                     weight_nr_accesses_permil=200,
-- 
2.47.3