Re: Memory tiering with DAMON/DAMOS auto-tuning

[SJ Park <sj@kernel.org>]

On Tue, 29 Sep 2026 16:54:41 +0300 Anton Gavriliuk <antosha20xx@gmail.com> wrote:

> > Also, fyi again, from damo v3.4.1, you can mark start of new kdamond, context,
> > target and scheme parameters on the command line using --kdamond, --damon_ctx,
> > --damon_target, and --damos_scheme.  you coud use those instead of
> > --damos_nr_quota_goals, --damos_nr_filters, --nr_targets, --nr_schemes and
> > --nr_ctxs.  E.g.,
> 
> Fedora shows old damo version,
> 
> [root@localhost anton]# rpm -qa|grep -i damo
> damo-3.3.0-1.fc44.noarch
> [root@localhost anton]#
> 
> so I configured latest available 3.4.1
> 
> [root@localhost damo]# damo version
> 3.4.1
> [root@localhost damo]# which damo
> /home/anton/damo/damo
> [root@localhost damo]#

Thank you!  Hopefully upgrading the version was not that difficult.  You can
simply git-clone the repo and use the 'damo' executable file under the
local-cloned repo.

> 
> > Seems you manually made this change.  Has this executed after 'damo start'?
> > Also, did you 'commit' the updated commit input?  If any of your answer to the
> > questions is not "yes", the tuner update may not applied.
> 
> Yes, it was executed before 'damo start', but I didn't do 'commit'.

To do this manually, you should write the files after 'damo start', and also do
'commit'.

Anyway, this means your previous run was using 'consist' tuner.  That explains
why it didn't show any difference.

> 
> > It would also be helpful if you could run 'damo report damon --damos_stats'
> > periodically (say, once per 5-10 seconds) while the migration is ongoing and
> > share the outputs with us.
> 
> Ok, initially I have,
> 
> [root@localhost ~]# numastat -z -p $(pgrep valkey-server)
> 
> Per-node process memory usage (in MBs) for PID 25361 (valkey-server)
>                            Node 0          Node 1          Node 2
>                   --------------- --------------- ---------------
> Heap                         0.11            0.00            0.00
> Stack                        0.03            0.00            0.00
> Private                 238356.60            0.50            4.86
> ----------------  --------------- --------------- ---------------
> Total                   238356.73            0.50            4.86
> 
>                             Total
>                   ---------------
> Heap                         0.11
> Stack                        0.03
> Private                 238361.95
> ----------------  ---------------
> Total                   238362.09
> [root@localhost ~]#
> 
> I run
> 
> [root@localhost ~]# /home/anton/damo/damo start \
>         ` # A kdamond to demote cold memory from node 0 to node 2 ` \
>         --kdamond --damon_ctx --monitoring_intervals_autotune \
>                 --damon_target --numa_node 0 \
>                 --damos_scheme \
>                         --damos_action migrate_cold 2 \
>                         --damos_access_rate 0% 0% --damos_apply_interval 1s \
>                         ` # up to 8 GiB per second ` \
>                         --damos_quota_interval 1s --damos_quota_space 8G \
>                         ` # aiming at least 50% node 0 free memory ` \
>                         --damos_quota_goal node_mem_free_bp 50% 0 \
>                         ` # using temporal tuner ` \
>                         --damos_quota_goal_tuner temporal \
>                         --damos_filter reject young \
>         ` # A kdamond to promote hot memory from node 2 to node 0 ` \
>         --kdamond --damon_ctx --monitoring_intervals_autotune \
>                 --damon_target --numa_node 2 \
>                 --damos_scheme \
>                         --damos_action migrate_hot 0 \
>                         --damos_access_rate 5% max --damos_apply_interval 1s \
>                         ` # up to 8 GiB per second ` \
>                         --damos_quota_interval 1s --damos_quota_space 8G \
>                         ` # aiming at least 50.3% node 0 memory utilization ` \
>                         --damos_quota_goal node_mem_used_bp 50.3% 0 \
>                         ` # using temporal tuner ` \
>                         --damos_quota_goal_tuner temporal \
>                         --damos_filter allow young \
> >
> sysinfo loading fail (info update fail (sysfs feature check fail
> (feature map making fail (staging damos goal feature check purpose
> kdamond failed))))

Oops...  Seems you were running next branch of damo.  There was a bug.  I
reproduced it on 7.2.8 kernel, and fixed it.  The fix [1] is now pushed.  Could
you pull the 'next' branch and try again?

> [root@localhost ~]#
> [root@localhost ~]# ps -ef|grep -i damo
> root       26051   25396  0 16:49 pts/2    00:00:00 grep --color=auto -i damo
> [root@localhost ~]#
> 
> > It would also be helpful if you could run 'damo report damon --damos_stats'
> > periodically (say, once per 5-10 seconds) while the migration is ongoing and
> > share the outputs with us.
> 
> Sure, I will start something like that,
> while true; do damo report damon --damos_stats >>
> /home/anton/damo_report_damon_damos_stats; sleep 10; done
> once damo will be started.

Sounds good.

> 
> 
> > Interesting.  I doubt if the tuner change is correctly made.  'damo report
> > damon' can help us understand under what configuration DAMON is running.  It
> > could help us quickly see if the configuration is done as we intended.  Could
> > you share 'damo report damon' result on the final state?
> 
> There are errors above during damo start.

Apparently it was a bug in damo's next branch.  As I mentioned above, the fix
is now pushed.  Could you try again?

[1] https://github.com/damonitor/damo/commit/fd2fb55b44ba03db55b8a025eef43dcf569831e5

Thanks,
SJ

[...]