* Issues getting per-numa heatmap via "paddr" tracking
@ 2023-06-24 23:26 Vinicius Petrucci
2023-07-13 17:14 ` SeongJae Park
0 siblings, 1 reply; 2+ messages in thread
From: Vinicius Petrucci @ 2023-06-24 23:26 UTC (permalink / raw)
To: damon
Hi,
I've cloned the latest damo user-space tool to try generating a
heatmap on a per-NUMA basis using DAMON's "paddr" feature.
I wrote a simple script using "damo" and "masim" as workload
controlled (via numactl) to use NUMA 0 for both CPU and memory
allocation, but I am not seeing any accesses on the target NUMA region
(say, NUMA 0)
I analyzed the debug output of the damo tool and I could see that at
least the _initial_ region was created macthing the NUMA address range
in my system.
write '4294967296' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/0/start'
write '824633720831' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/0/end'
# lsmem -r -o NODE,RANGE,ZONES,SIZE
NODE RANGE ZONES SIZE
0 0x0000000100000000-0x000000bfffffffff Normal 764G
>>> 0x0000000100000000
4294967296
>>> 0x000000bfffffffff
824633720831
So, I am not sure what is happening. Thoughts?
Am I missing anything or doing anything wrong?
(See scripts and results below)
Thanks!
------------
System details:
# uname -a
Linux 6.1.35
# perf version
perf version 6.3.9-1.el9.elrepo.x86_64
# damo features
fvaddr: Supported
init_regions: Supported
init_regions_target_idx: Supported
paddr: Supported
record: Unsupported
schemes: Supported
schemes_filters: Unsupported
schemes_prioritization: Supported
schemes_quotas: Supported
schemes_speed_limit: Supported
schemes_stat_qt_exceed: Supported
schemes_stat_succ: Supported
schemes_tried_regions: Unsupported
schemes_tried_regions_sz: Unsupported
schemes_wmarks: Supported
vaddr: Supported
# lsmem -r -o NODE,RANGE,ZONES,SIZE
NODE RANGE ZONES SIZE
0 0x0000000100000000-0x000000bfffffffff Normal 764G
1 0x000000c080000000-0x000000fc7fffffff Normal 240G
1 0x0000010000000000-0x00000183ffffffff Normal 528G
----- BASH ------
numactl -N 0 -m 0 /opt/masim/masim
/opt/masim/configs/stairs_30secs.cfg 1> stairs.log & masim_pid=$!
/opt/damo/damo record paddr --debug_damon --numa_node 0 1> damo.log &
damo_pid=$!
wait $masim_pid
kill $damo_pid
sleep 5
wait $damo_pid
/opt/damo/damo report heats --heatmap stdout
--------- OUTPUT ----
[ perf record: Woken up 4 times to write data ]
[ perf record: Captured and wrote 1.731 MB damon.data (10601 samples) ]
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
# access_frequency: 0 1 2 3 4 5 6 7 8 9
# x-axis: space (4294967296-826781204416: 766.000 GiB)
# y-axis: time (26583584434000-26611562269000: 27.978 s)
# resolution: 80x40 (9.575 GiB and 699.446 ms for each character)
# cat stairs.log
initial phase: 293,863 accesses/msec, 5000 msecs run
phase 0: 293,915 accesses/msec, 2500 msecs run
phase 1: 290,245 accesses/msec, 2500 msecs run
phase 2: 293,968 accesses/msec, 2500 msecs run
phase 3: 292,185 accesses/msec, 2500 msecs run
phase 4: 305,397 accesses/msec, 2500 msecs run
phase 5: 292,657 accesses/msec, 2500 msecs run
phase 6: 290,874 accesses/msec, 2500 msecs run
phase 7: 291,556 accesses/msec, 2500 msecs run
phase 8: 304,506 accesses/msec, 2500 msecs run
phase 9: 292,709 accesses/msec, 2500 msecs run
# cat damo.log
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/nr_contexts': '1'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/monitoring_attrs/nr_regions/min':
'3'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/monitoring_attrs/nr_regions/max':
'3'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/monitoring_attrs/intervals/sample_us':
'10000'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/monitoring_attrs/intervals/aggr_us':
'100000'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/monitoring_attrs/intervals/update_us':
'1000000'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/operations': 'paddr'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/nr_targets': '1'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/nr_regions':
'3'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/1/end':
'1084479242239'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/1/start':
'826781204480'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/2/end':
'1666447310847'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/2/start':
'1099511627776'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/0/end':
'824633720831'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/0/start':
'4294967296'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/pid_target':
'0'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/schemes/nr_schemes': '0'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/avail_operations': 'vaddr
fvaddr
paddr'
read '/sys/kernel/mm/damon/admin/kdamonds/0/state': 'off'
read '/sys/kernel/mm/damon/admin/kdamonds/0/pid': '-1'
read '/sys/kernel/mm/damon/admin/kdamonds/nr_kdamonds': '1'
read '/sys/kernel/mm/damon/admin/kdamonds/nr_kdamonds': '1'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/nr_contexts': '1'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/nr_targets': '1'
write '0' to '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/nr_targets'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/schemes/nr_schemes': '0'
write '1' to '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/schemes/nr_schemes'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/avail_operations': 'vaddr
fvaddr
paddr'
read '/sys/kernel/mm/damon/admin/kdamonds/nr_kdamonds': '1'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/nr_contexts': '1'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/nr_targets': '0'
write '1' to '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/nr_targets'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/nr_regions':
'0'
write '3' to '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/nr_regions'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/schemes/nr_schemes': '1'
write '0' to '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/schemes/nr_schemes'
write 'paddr' to '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/operations'
write '10000' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/monitoring_attrs/intervals/sample_us'
write '100000' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/monitoring_attrs/intervals/aggr_us'
write '1000000' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/monitoring_attrs/intervals/update_us'
write '3' to '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/monitoring_attrs/nr_regions/min'
write '3' to '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/monitoring_attrs/nr_regions/max'
write '' to '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/pid_target'
write '4294967296' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/0/start'
write '824633720831' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/0/end'
write '826781204480' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/1/start'
write '1084479242239' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/1/end'
write '1099511627776' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/2/start'
write '1666447310847' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/2/end'
read '/sys/kernel/mm/damon/admin/kdamonds/nr_kdamonds': '1'
read '/sys/kernel/mm/damon/admin/kdamonds/0/state': 'off'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/nr_contexts': '1'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/monitoring_attrs/nr_regions/min':
'3'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/monitoring_attrs/nr_regions/max':
'3'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/monitoring_attrs/intervals/sample_us':
'10000'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/monitoring_attrs/intervals/aggr_us':
'100000'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/monitoring_attrs/intervals/update_us':
'1000000'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/operations': 'paddr'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/nr_targets': '1'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/nr_regions':
'3'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/1/end':
'1084479242239'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/1/start':
'826781204480'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/2/end':
'1666447310847'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/2/start':
'1099511627776'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/0/end':
'824633720831'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/0/start':
'4294967296'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/pid_target':
'0'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/schemes/nr_schemes': '0'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/avail_operations': 'vaddr
fvaddr
paddr'
read '/sys/kernel/mm/damon/admin/kdamonds/0/state': 'off'
read '/sys/kernel/mm/damon/admin/kdamonds/0/pid': '-1'
read '/sys/kernel/mm/damon/admin/kdamonds/nr_kdamonds': '1'
read '/sys/kernel/mm/damon/admin/kdamonds/nr_kdamonds': '1'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/nr_contexts': '1'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/nr_targets': '1'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/nr_regions':
'3'
write '28' to '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/nr_regions'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/schemes/nr_schemes': '0'
write 'paddr' to '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/operations'
write '5000' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/monitoring_attrs/intervals/sample_us'
write '100000' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/monitoring_attrs/intervals/aggr_us'
write '1000000' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/monitoring_attrs/intervals/update_us'
write '10' to '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/monitoring_attrs/nr_regions/min'
write '1000' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/monitoring_attrs/nr_regions/max'
write '' to '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/pid_target'
write '4096' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/0/start'
write '655360' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/0/end'
write '1048576' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/1/start'
write '1975517184' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/1/end'
write '1976205312' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/2/start'
write '1976565760' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/2/end'
write '1979711488' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/3/start'
write '2567876608' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/3/end'
write '2567880704' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/4/start'
write '2569625624' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/4/end'
write '2569625624' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/5/start'
write '2570024536' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/5/end'
write '2570024536' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/6/start'
write '2570027032' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/6/end'
write '2570027032' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/7/start'
write '2570425944' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/7/end'
write '2570425944' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/8/start'
write '2570428440' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/8/end'
write '2570428440' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/9/start'
write '2570494040' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/9/end'
write '2570494040' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/10/start'
write '2570498072' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/10/end'
write '2570498072' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/11/start'
write '2570562648' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/11/end'
write '2570562648' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/12/start'
write '2570563608' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/12/end'
write '2570563608' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/13/start'
write '2570623064' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/13/end'
write '2570623064' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/14/start'
write '2570625048' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/14/end'
write '2570625048' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/15/start'
write '2570686040' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/15/end'
write '2570686040' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/16/start'
write '2570686488' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/16/end'
write '2570686488' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/17/start'
write '2570751064' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/17/end'
write '2570751064' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/18/start'
write '2570752024' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/18/end'
write '2570752024' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/19/start'
write '2570816600' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/19/end'
write '2570816600' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/20/start'
write '2570817560' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/20/end'
write '2570817560' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/21/start'
write '2571045464' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/21/end'
write '2571045464' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/22/start'
write '2571046936' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/22/end'
write '2571046936' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/23/start'
write '2571274840' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/23/end'
write '2571274840' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/24/start'
write '2594201600' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/24/end'
write '2596564992' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/25/start'
write '2716004352' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/25/end'
write '2787110912' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/26/start'
write '2818560000' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/26/end'
write '4294967296' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/27/start'
write '826781204480' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/27/end'
write 'on' to '/sys/kernel/mm/damon/admin/kdamonds/0/state'
read '/sys/kernel/mm/damon/admin/kdamonds/nr_kdamonds': '1'
read '/sys/kernel/mm/damon/admin/kdamonds/0/state': 'on'
Press Ctrl+C to stop
read '/sys/kernel/mm/damon/admin/kdamonds/nr_kdamonds': '1'
read '/sys/kernel/mm/damon/admin/kdamonds/0/state': 'on'
read '/sys/kernel/mm/damon/admin/kdamonds/0/state': 'on'
read '/sys/kernel/mm/damon/admin/kdamonds/0/state': 'on'
read '/sys/kernel/mm/damon/admin/kdamonds/0/state': 'on'
read '/sys/kernel/mm/damon/admin/kdamonds/0/state': 'on'
read '/sys/kernel/mm/damon/admin/kdamonds/0/state': 'on'
read '/sys/kernel/mm/damon/admin/kdamonds/0/state': 'on'
read '/sys/kernel/mm/damon/admin/kdamonds/0/state': 'on'
read '/sys/kernel/mm/damon/admin/kdamonds/0/state': 'on'
read '/sys/kernel/mm/damon/admin/kdamonds/0/state': 'on'
read '/sys/kernel/mm/damon/admin/kdamonds/0/state': 'on'
read '/sys/kernel/mm/damon/admin/kdamonds/0/state': 'on'
read '/sys/kernel/mm/damon/admin/kdamonds/0/state': 'on'
read '/sys/kernel/mm/damon/admin/kdamonds/0/state': 'on'
read '/sys/kernel/mm/damon/admin/kdamonds/0/state': 'on'
read '/sys/kernel/mm/damon/admin/kdamonds/0/state': 'on'
read '/sys/kernel/mm/damon/admin/kdamonds/0/state': 'on'
read '/sys/kernel/mm/damon/admin/kdamonds/0/state': 'on'
read '/sys/kernel/mm/damon/admin/kdamonds/0/state': 'on'
read '/sys/kernel/mm/damon/admin/kdamonds/0/state': 'on'
read '/sys/kernel/mm/damon/admin/kdamonds/0/state': 'on'
read '/sys/kernel/mm/damon/admin/kdamonds/0/state': 'on'
read '/sys/kernel/mm/damon/admin/kdamonds/0/state': 'on'
read '/sys/kernel/mm/damon/admin/kdamonds/0/state': 'on'
read '/sys/kernel/mm/damon/admin/kdamonds/0/state': 'on'
read '/sys/kernel/mm/damon/admin/kdamonds/0/state': 'on'
read '/sys/kernel/mm/damon/admin/kdamonds/0/state': 'on'
read '/sys/kernel/mm/damon/admin/kdamonds/0/state': 'on'
read '/sys/kernel/mm/damon/admin/kdamonds/0/state': 'on'
signal 15 received
write 'off' to '/sys/kernel/mm/damon/admin/kdamonds/0/state'
read '/sys/kernel/mm/damon/admin/kdamonds/nr_kdamonds': '1'
read '/sys/kernel/mm/damon/admin/kdamonds/0/state': 'off'
read '/sys/kernel/mm/damon/admin/kdamonds/nr_kdamonds': '1'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/nr_contexts': '1'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/nr_targets': '1'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/nr_regions':
'28'
write '3' to '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/nr_regions'
read '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/schemes/nr_schemes': '0'
write 'paddr' to '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/operations'
write '10000' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/monitoring_attrs/intervals/sample_us'
write '100000' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/monitoring_attrs/intervals/aggr_us'
write '1000000' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/monitoring_attrs/intervals/update_us'
write '3' to '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/monitoring_attrs/nr_regions/min'
write '3' to '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/monitoring_attrs/nr_regions/max'
write '' to '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/pid_target'
write '4294967296' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/0/start'
write '824633720831' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/0/end'
write '826781204480' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/1/start'
write '1084479242239' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/1/end'
write '1099511627776' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/2/start'
write '1666447310847' to
'/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/2/end'
^ permalink raw reply [flat|nested] 2+ messages in thread
* Re: Issues getting per-numa heatmap via "paddr" tracking
2023-06-24 23:26 Issues getting per-numa heatmap via "paddr" tracking Vinicius Petrucci
@ 2023-07-13 17:14 ` SeongJae Park
0 siblings, 0 replies; 2+ messages in thread
From: SeongJae Park @ 2023-07-13 17:14 UTC (permalink / raw)
To: Vinicius Petrucci; +Cc: damon, sj
Hi Vinicius,
On Sat, 24 Jun 2023 18:26:34 -0500 Vinicius Petrucci <vpetrucci@gmail.com> wrote:
Sorry for late response. Somehow I missed this mail.
> Hi,
>
> I've cloned the latest damo user-space tool to try generating a
> heatmap on a per-NUMA basis using DAMON's "paddr" feature.
> I wrote a simple script using "damo" and "masim" as workload
> controlled (via numactl) to use NUMA 0 for both CPU and memory
> allocation, but I am not seeing any accesses on the target NUMA region
> (say, NUMA 0)
Great.
>
> I analyzed the debug output of the damo tool and I could see that at
> least the _initial_ region was created macthing the NUMA address range
> in my system.
>
> write '4294967296' to
> '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/0/start'
> write '824633720831' to
> '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/0/end'
>
> # lsmem -r -o NODE,RANGE,ZONES,SIZE
> NODE RANGE ZONES SIZE
> 0 0x0000000100000000-0x000000bfffffffff Normal 764G
>
> >>> 0x0000000100000000
> 4294967296
> >>> 0x000000bfffffffff
> 824633720831
>
> So, I am not sure what is happening. Thoughts?
> Am I missing anything or doing anything wrong?
> (See scripts and results below)
Thank you for this great and nice report! I left my comment below.
>
> Thanks!
>
> ------------
>
> System details:
> # uname -a
> Linux 6.1.35
>
> # perf version
> perf version 6.3.9-1.el9.elrepo.x86_64
>
> # damo features
> fvaddr: Supported
> init_regions: Supported
> init_regions_target_idx: Supported
> paddr: Supported
> record: Unsupported
> schemes: Supported
> schemes_filters: Unsupported
> schemes_prioritization: Supported
> schemes_quotas: Supported
> schemes_speed_limit: Supported
> schemes_stat_qt_exceed: Supported
> schemes_stat_succ: Supported
> schemes_tried_regions: Unsupported
> schemes_tried_regions_sz: Unsupported
> schemes_wmarks: Supported
> vaddr: Supported
>
> # lsmem -r -o NODE,RANGE,ZONES,SIZE
> NODE RANGE ZONES SIZE
> 0 0x0000000100000000-0x000000bfffffffff Normal 764G
> 1 0x000000c080000000-0x000000fc7fffffff Normal 240G
> 1 0x0000010000000000-0x00000183ffffffff Normal 528G
>
> ----- BASH ------
>
> numactl -N 0 -m 0 /opt/masim/masim
> /opt/masim/configs/stairs_30secs.cfg 1> stairs.log & masim_pid=$!
> /opt/damo/damo record paddr --debug_damon --numa_node 0 1> damo.log &
> damo_pid=$!
> wait $masim_pid
> kill $damo_pid
> sleep 5
> wait $damo_pid
> /opt/damo/damo report heats --heatmap stdout
Nice! I hope to have this kind of scripts in tests/ directory of 'damo'.
>
> --------- OUTPUT ----
>
> [ perf record: Woken up 4 times to write data ]
> [ perf record: Captured and wrote 1.731 MB damon.data (10601 samples) ]
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> # access_frequency: 0 1 2 3 4 5 6 7 8 9
> # x-axis: space (4294967296-826781204416: 766.000 GiB)
> # y-axis: time (26583584434000-26611562269000: 27.978 s)
> # resolution: 80x40 (9.575 GiB and 699.446 ms for each character)
As the last four lines of the output say, this output shows when (y-axis) which
regions (x-axis) are how frequently accessed (number), using 80 characters for
each region, and 40 characters for the entime time. The size of the memory
region that monitored and displayed here is about 766 GiB. The entire time of
the monitoring is similar to the runtime of 'masim', which is about 30 seconds.
Hence, each character shows the monitored access pattern of about 9.5 GiB and
699.446 ms, as above '# resolution' line says.
Meanwhile, the 'masim' run would accessed only 10MB region at one time. Since
that's quite tiny amount of memory compared to the total monitored memory
region (766 GiB) and even compared to the single character of the output (9.5
GiB) due to the low resolution (80x40), the output couldn't show the access.
In short, you are visualizing memory region that very large compared to the
region that actual access has made, with too low resolution to show the small
regions of accesses.
I think you could use 'damo report raw' like command to show low level
monitoring results, or 'damo report wss' to show only regions shown accesses.
Thanks,
SJ
[...]
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2023-07-13 17:14 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-06-24 23:26 Issues getting per-numa heatmap via "paddr" tracking Vinicius Petrucci
2023-07-13 17:14 ` SeongJae Park
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.