Re: [LSF/MM/BPF TOPIC] Using hardware counters to determine hot/cold pages

[SeongJae Park <sj@kernel.org>]

Hi Aneesh,

On Fri, 17 Feb 2023 17:28:09 +0530 Aneesh Kumar K V <aneesh.kumar@linux.ibm.com> wrote:

> PowerPC architecture (POWER10) supports a Hot/Cold page tracking
> facility that provides access counter and access affinity details at
> configurable page size granularity [1]. I have been looking at using
> this counter in different areas of the kernel such as
> 
> 1) Page reclaim/demotion
> 2) THP utilization
> 3) Page promotion.
> 
> I have done some MGLRU integration and would like to discuss the
> observation with the rest of the community. It is still not clear what
> are the best ways to integrate these hardware counters in the Linux
> kernel.

Sounds very interesting.  I think DAMON might be one another option, because it
is designed to be easy to extended with various source of access
information[1], and provides an abstraction layer for access temparature based
memory management[2], namely Data Access Monitoring-based Operation Schemes
(DAMOS).

> Attached is the performance graph showing how the mongodb/ycsb
> benchmark performs when using hardware counters with MGLRU aging. An
> early RFC version of the code can be found at
> https://github.com/kvaneesh/linux/commit/b472e2c8080823bb4114c286270aea3e18ffe221
> . I also expect we can get some numbers w.r.t THP usage before the
> conference.

I also have experimented a DAMON-based THP optimization[3], which shown
interesting results.

Hope to discuss about this with you at LSF/MM.  FYI, I also proposed an LSF/MM
topic for DAMON[4].

[1] https://docs.kernel.org/mm/damon/design.html#configurable-layers
[2] https://docs.kernel.org/mm/damon/api.html#c.damos
[3] https://www.amazon.science/publications/daos-data-access-aware-operating-system
[4] https://lore.kernel.org/damon/20230214003328.55285-1-sj@kernel.org/


Thanks,
SJ

> 
> 
> X axis is the amount of memory that I am removing from the system so
> that I can force more memory reclaims. The total memory available is
> 50GB/single NUMA node/64 CPUs,40GB database with 40GB cache
> configuration.
> 
> 
> [1]
> https://hc32.hotchips.org/assets/program/conference/day1/HotChips2020_Server_Processors_IBM_Starke_POWER10_v33.pdf
>