Re: [RFC PATCH v1] tools/mm: Add thpmaps script to dump THP usage info

Barry Song
2024-01-10T03:51:54Z

On Wed, Jan 10, 2024 at 11:35 AM John Hubbard <jhubbard@nvidia.com> wrote:
>
> On 1/2/24 07:38, Ryan Roberts wrote:
> > With the proliferation of large folios for file-backed memory, and more
> > recently the introduction of multi-size THP for anonymous memory, it is
> > becoming useful to be able to see exactly how large folios are mapped
> > into processes. For some architectures (e.g. arm64), if most memory is
> > mapped using contpte-sized and -aligned blocks, TLB usage can be
> > optimized so it's useful to see where these requirements are and are not
> > being met.
> >
> > thpmaps is a Python utility that reads /proc/<pid>/smaps,
> > /proc/<pid>/pagemap and /proc/kpageflags to print information about how
> > transparent huge pages (both file and anon) are mapped to a specified
> > process or cgroup. It aims to help users debug and optimize their
> > workloads. In future we may wish to introduce stats directly into the
> > kernel (e.g. smaps or similar), but for now this provides a short term
> > solution without the need to introduce any new ABI.
> >
> ...
> > I've found this very useful for debugging, and I know others have requested a
> > way to check if mTHP and contpte is working, so thought this might a good short
> > term solution until we figure out how best to add stats in the kernel?
> >
>
> Hi Ryan,
>
> One thing that immediately came up during some recent testing of mTHP
> on arm64: the pid requirement is sometimes a little awkward. I'm running
> tests on a machine at a time for now, inside various containers and
> such, and it would be nice if there were an easy way to get some numbers
> for the mTHPs across the whole machine.
>
> I'm not sure if that changes anything about thpmaps here. Probably
> this is fine as-is. But I wanted to give some initial reactions from
> just some quick runs: the global state would be convenient.

+1. but this seems to be impossible by scanning pagemap?
so may we add this statistics information in kernel just like
/proc/meminfo or a separate /proc/mthp_info?

>
> thanks,
> --
> John Hubbard
> NVIDIA

Thanks
barry