Re: [PATCH v4 0/3] mm: process/cgroup ksm support

[Mike Kravetz <mike.kravetz@oracle.com>]

On 03/15/23 21:03, David Hildenbrand wrote:
> On 10.03.23 19:28, Stefan Roesch wrote:
> 
> Stefan, can you do me a favor and investigate which pages we end up
> deduplicating -- especially if it's mostly only the zeropage and if it's
> still that significant when disabling THP?
> 
> I'm currently investigating with some engineers on playing with enabling KSM
> on some selected processes (enabling it blindly on all VMAs of that process
> via madvise() ).
> 
> One thing we noticed is that such (~50 times) 20MiB processes end up saving
> ~2MiB of memory per process. That made me suspicious, because it's the THP
> size.
> 
> What I think happens is that we have a 2 MiB area (stack?) and only touch a
> single page. We get a whole 2 MiB THP populated. Most of that THP is zeroes.
> 
> KSM somehow ends up splitting that THP and deduplicates all resulting
> zeropages. Thus, we "save" 2 MiB. Actually, it's more like we no longer
> "waste" 2 MiB. I think the processes with KSM have less (none) THP than the
> processes with THP enabled, but I only took a look at a sample of the
> process' smaps so far.
> 
> I recall that there was a proposal to split underutilized THP and free up
> the zeropages (IIRC Rik was involved).
> 
> I also recall that Mike reported memory waste due to THP.

Interesting!

2MB stacks were also involved in our case.  That stack would first get a
write fault allocating a THP.  The write fault would be followed by a
mprotect(PROT_NONE) of the 4K page at the bottom of the stack to create
a guard page.  The mprotect would result in the THP being split resulting
in 510 zero filled pages.  I suppose KSM could dedup those zero pages.
-- 
Mike Kravetz