Re: Slow-tier Page Promotion discussion recap and open questions

SeongJae Park
2024-12-18T19:23:54Z

On Tue, 17 Dec 2024 20:19:56 -0800 (PST) David Rientjes <rientjes@google.com> wrote:

> Hi everybody,
>
> We had a very interactive discussion last week led by RaghavendraKT on
> slow-tier page promotion intended for memory tiering platforms, thank
> you! Thanks as well to everybody who attended and provided great
> questions, suggestions, and feedback.
>
> The RFC patch series "mm: slowtier page promotion based on PTE A bit"[1]
> is a proposal to allow for asynchronous page promotion based on memory
> accesses as an alternative to NUMA Balancing based promotions. There was
> widespread interest in this topic and the discussion surfaced multiple
> use cases and requirements, very focused on CXL use cases.

Thank you for keeping the series and this great summary, David :)

[...]
> ----->o-----
> I followed up on a discussion point early in the talk about whether this
> should be virtual address scanning like the current approach, walking
> mm_struct's, or the alternative approach which would be physical address
> scanning.
>
> Raghu sees this as a fully alternative approach such as what DAMON uses
> that is based on rmap. The only advantage appears to be avoiding
> scanning on top tier memory completely.

IMHO, there could be more advantages of physical address space based
appraoches. Easier handling of unmapped pages and short-lived processes,
applying different access monitoring / promotion policies for differnt NUMA
nodes (tiers) are some of those off the top of my head.

>
> ----->o-----
> Wei noted there was a lot of similarities between the RFC implementation
> and the MGLRU page walk functionality and whether it would make sense to
> try to converge these together or make more generally useful.
>
> SeongJae noted that if DAMON logic were used for the scanning that we
> could re-use the existing support for controlling the overhead.

Just to clarify. I added this comment since there were concerns around rmap
overhead for pysical address space-based monitoring approaches.

[...]
> My takeaways:
[...]
> - I think virtual memory scanning is likely the only viable approach for
> this purpose and we could store state in the underlying struct page,
> similar to NUMA Balancing, but that all scanning should be driven by
> walking the mm_struct's to harvest the Accessed bit

I don't clearly get why you think virtual memory scanning is the only viable
approach. I'm curious if you have some pros/cons list about virtual vs
physical address based appraoches in your mind, and willing to share.

[...]
> We'll be looking to incorporate this discussion in our upstream Memory
> Tiering Working Group to accelerate alignment and progress on the
> approach.

Thank you again for your efforts on this!

Thanks,
SJ …