From: SeongJae Park <sj@kernel.org>
To: Vinicius Petrucci <vpetrucci@gmail.com>
Cc: damon@lists.linux.dev, sj@kernel.org
Subject: Re: Issues getting per-numa heatmap via "paddr" tracking
Date: Thu, 13 Jul 2023 17:14:35 +0000 [thread overview]
Message-ID: <20230713171435.66187-1-sj@kernel.org> (raw)
In-Reply-To: <CAEZ6=UPwe8jUD4B4G5n8drBf3HieQ=-KsY6fs2MPh0xtQxeMgg@mail.gmail.com>
Hi Vinicius,
On Sat, 24 Jun 2023 18:26:34 -0500 Vinicius Petrucci <vpetrucci@gmail.com> wrote:
Sorry for late response. Somehow I missed this mail.
> Hi,
>
> I've cloned the latest damo user-space tool to try generating a
> heatmap on a per-NUMA basis using DAMON's "paddr" feature.
> I wrote a simple script using "damo" and "masim" as workload
> controlled (via numactl) to use NUMA 0 for both CPU and memory
> allocation, but I am not seeing any accesses on the target NUMA region
> (say, NUMA 0)
Great.
>
> I analyzed the debug output of the damo tool and I could see that at
> least the _initial_ region was created macthing the NUMA address range
> in my system.
>
> write '4294967296' to
> '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/0/start'
> write '824633720831' to
> '/sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/regions/0/end'
>
> # lsmem -r -o NODE,RANGE,ZONES,SIZE
> NODE RANGE ZONES SIZE
> 0 0x0000000100000000-0x000000bfffffffff Normal 764G
>
> >>> 0x0000000100000000
> 4294967296
> >>> 0x000000bfffffffff
> 824633720831
>
> So, I am not sure what is happening. Thoughts?
> Am I missing anything or doing anything wrong?
> (See scripts and results below)
Thank you for this great and nice report! I left my comment below.
>
> Thanks!
>
> ------------
>
> System details:
> # uname -a
> Linux 6.1.35
>
> # perf version
> perf version 6.3.9-1.el9.elrepo.x86_64
>
> # damo features
> fvaddr: Supported
> init_regions: Supported
> init_regions_target_idx: Supported
> paddr: Supported
> record: Unsupported
> schemes: Supported
> schemes_filters: Unsupported
> schemes_prioritization: Supported
> schemes_quotas: Supported
> schemes_speed_limit: Supported
> schemes_stat_qt_exceed: Supported
> schemes_stat_succ: Supported
> schemes_tried_regions: Unsupported
> schemes_tried_regions_sz: Unsupported
> schemes_wmarks: Supported
> vaddr: Supported
>
> # lsmem -r -o NODE,RANGE,ZONES,SIZE
> NODE RANGE ZONES SIZE
> 0 0x0000000100000000-0x000000bfffffffff Normal 764G
> 1 0x000000c080000000-0x000000fc7fffffff Normal 240G
> 1 0x0000010000000000-0x00000183ffffffff Normal 528G
>
> ----- BASH ------
>
> numactl -N 0 -m 0 /opt/masim/masim
> /opt/masim/configs/stairs_30secs.cfg 1> stairs.log & masim_pid=$!
> /opt/damo/damo record paddr --debug_damon --numa_node 0 1> damo.log &
> damo_pid=$!
> wait $masim_pid
> kill $damo_pid
> sleep 5
> wait $damo_pid
> /opt/damo/damo report heats --heatmap stdout
Nice! I hope to have this kind of scripts in tests/ directory of 'damo'.
>
> --------- OUTPUT ----
>
> [ perf record: Woken up 4 times to write data ]
> [ perf record: Captured and wrote 1.731 MB damon.data (10601 samples) ]
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> # access_frequency: 0 1 2 3 4 5 6 7 8 9
> # x-axis: space (4294967296-826781204416: 766.000 GiB)
> # y-axis: time (26583584434000-26611562269000: 27.978 s)
> # resolution: 80x40 (9.575 GiB and 699.446 ms for each character)
As the last four lines of the output say, this output shows when (y-axis) which
regions (x-axis) are how frequently accessed (number), using 80 characters for
each region, and 40 characters for the entime time. The size of the memory
region that monitored and displayed here is about 766 GiB. The entire time of
the monitoring is similar to the runtime of 'masim', which is about 30 seconds.
Hence, each character shows the monitored access pattern of about 9.5 GiB and
699.446 ms, as above '# resolution' line says.
Meanwhile, the 'masim' run would accessed only 10MB region at one time. Since
that's quite tiny amount of memory compared to the total monitored memory
region (766 GiB) and even compared to the single character of the output (9.5
GiB) due to the low resolution (80x40), the output couldn't show the access.
In short, you are visualizing memory region that very large compared to the
region that actual access has made, with too low resolution to show the small
regions of accesses.
I think you could use 'damo report raw' like command to show low level
monitoring results, or 'damo report wss' to show only regions shown accesses.
Thanks,
SJ
[...]
prev parent reply other threads:[~2023-07-13 17:14 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-06-24 23:26 Issues getting per-numa heatmap via "paddr" tracking Vinicius Petrucci
2023-07-13 17:14 ` SeongJae Park [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230713171435.66187-1-sj@kernel.org \
--to=sj@kernel.org \
--cc=damon@lists.linux.dev \
--cc=vpetrucci@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.