Re: [PATCH v2 01/13] mm: support madvise(MADV_FREE)

[Minchan Kim <minchan-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>]

On Thu, Nov 05, 2015 at 10:33:50AM +0900, Minchan Kim wrote:
> On Wed, Nov 04, 2015 at 12:00:06PM -0800, Shaohua Li wrote:
> > On Wed, Nov 04, 2015 at 10:25:55AM +0900, Minchan Kim wrote:
> > > Linux doesn't have an ability to free pages lazy while other OS already
> > > have been supported that named by madvise(MADV_FREE).
> > > 
> > > The gain is clear that kernel can discard freed pages rather than swapping
> > > out or OOM if memory pressure happens.
> > > 
> > > Without memory pressure, freed pages would be reused by userspace without
> > > another additional overhead(ex, page fault + allocation + zeroing).
> > > 
> > > Jason Evans said:
> > > 
> > > : Facebook has been using MAP_UNINITIALIZED
> > > : (https://lkml.org/lkml/2012/1/18/308) in some of its applications for
> > > : several years, but there are operational costs to maintaining this
> > > : out-of-tree in our kernel and in jemalloc, and we are anxious to retire it
> > > : in favor of MADV_FREE.  When we first enabled MAP_UNINITIALIZED it
> > > : increased throughput for much of our workload by ~5%, and although the
> > > : benefit has decreased using newer hardware and kernels, there is still
> > > : enough benefit that we cannot reasonably retire it without a replacement.
> > > :
> > > : Aside from Facebook operations, there are numerous broadly used
> > > : applications that would benefit from MADV_FREE.  The ones that immediately
> > > : come to mind are redis, varnish, and MariaDB.  I don't have much insight
> > > : into Android internals and development process, but I would hope to see
> > > : MADV_FREE support eventually end up there as well to benefit applications
> > > : linked with the integrated jemalloc.
> > > :
> > > : jemalloc will use MADV_FREE once it becomes available in the Linux kernel.
> > > : In fact, jemalloc already uses MADV_FREE or equivalent everywhere it's
> > > : available: *BSD, OS X, Windows, and Solaris -- every platform except Linux
> > > : (and AIX, but I'm not sure it even compiles on AIX).  The lack of
> > > : MADV_FREE on Linux forced me down a long series of increasingly
> > > : sophisticated heuristics for madvise() volume reduction, and even so this
> > > : remains a common performance issue for people using jemalloc on Linux.
> > > : Please integrate MADV_FREE; many people will benefit substantially.
> > > 
> > > How it works:
> > > 
> > > When madvise syscall is called, VM clears dirty bit of ptes of the range.
> > > If memory pressure happens, VM checks dirty bit of page table and if it
> > > found still "clean", it means it's a "lazyfree pages" so VM could discard
> > > the page instead of swapping out.  Once there was store operation for the
> > > page before VM peek a page to reclaim, dirty bit is set so VM can swap out
> > > the page instead of discarding.
> > > 
> > > Firstly, heavy users would be general allocators(ex, jemalloc, tcmalloc
> > > and hope glibc supports it) and jemalloc/tcmalloc already have supported
> > > the feature for other OS(ex, FreeBSD)
> > > 
> > > barrios@blaptop:~/benchmark/ebizzy$ lscpu
> > > Architecture:          x86_64
> > > CPU op-mode(s):        32-bit, 64-bit
> > > Byte Order:            Little Endian
> > > CPU(s):                12
> > > On-line CPU(s) list:   0-11
> > > Thread(s) per core:    1
> > > Core(s) per socket:    1
> > > Socket(s):             12
> > > NUMA node(s):          1
> > > Vendor ID:             GenuineIntel
> > > CPU family:            6
> > > Model:                 2
> > > Stepping:              3
> > > CPU MHz:               3200.185
> > > BogoMIPS:              6400.53
> > > Virtualization:        VT-x
> > > Hypervisor vendor:     KVM
> > > Virtualization type:   full
> > > L1d cache:             32K
> > > L1i cache:             32K
> > > L2 cache:              4096K
> > > NUMA node0 CPU(s):     0-11
> > > ebizzy benchmark(./ebizzy -S 10 -n 512)
> > > 
> > > Higher avg is better.
> > > 
> > >  vanilla-jemalloc		MADV_free-jemalloc
> > > 
> > > 1 thread
> > > records: 10			    records: 10
> > > avg:	2961.90			    avg:   12069.70
> > > std:	  71.96(2.43%)		    std:     186.68(1.55%)
> > > max:	3070.00			    max:   12385.00
> > > min:	2796.00			    min:   11746.00
> > > 
> > > 2 thread
> > > records: 10			    records: 10
> > > avg:	5020.00			    avg:   17827.00
> > > std:	 264.87(5.28%)		    std:     358.52(2.01%)
> > > max:	5244.00			    max:   18760.00
> > > min:	4251.00			    min:   17382.00
> > > 
> > > 4 thread
> > > records: 10			    records: 10
> > > avg:	8988.80			    avg:   27930.80
> > > std:	1175.33(13.08%)		    std:    3317.33(11.88%)
> > > max:	9508.00			    max:   30879.00
> > > min:	5477.00			    min:   21024.00
> > > 
> > > 8 thread
> > > records: 10			    records: 10
> > > avg:   13036.50			    avg:   33739.40
> > > std:	 170.67(1.31%)		    std:    5146.22(15.25%)
> > > max:   13371.00			    max:   40572.00
> > > min:   12785.00			    min:   24088.00
> > > 
> > > 16 thread
> > > records: 10			    records: 10
> > > avg:   11092.40			    avg:   31424.20
> > > std:	 710.60(6.41%)		    std:    3763.89(11.98%)
> > > max:   12446.00			    max:   36635.00
> > > min:	9949.00			    min:   25669.00
> > > 
> > > 32 thread
> > > records: 10			    records: 10
> > > avg:   11067.00			    avg:   34495.80
> > > std:	 971.06(8.77%)		    std:    2721.36(7.89%)
> > > max:   12010.00			    max:   38598.00
> > > min:	9002.00			    min:   30636.00
> > > 
> > > In summary, MADV_FREE is about much faster than MADV_DONTNEED.
> > 
> > The MADV_FREE is discussed for a while, it probably is too late to propose
> > something new, but we had the new idea (from Ben Maurer, CCed) recently and
> > think it's better. Our target is still jemalloc.
> > 
> > Compared to MADV_DONTNEED, MADV_FREE's lazy memory free is a huge win to reduce
> > page fault. But there is one issue remaining, the TLB flush. Both MADV_DONTNEED
> > and MADV_FREE do TLB flush. TLB flush overhead is quite big in contemporary
> > multi-thread applications. In our production workload, we observed 80% CPU
> > spending on TLB flush triggered by jemalloc madvise(MADV_DONTNEED) sometimes.
> > We haven't tested MADV_FREE yet, but the result should be similar. It's hard to
> > avoid the TLB flush issue with MADV_FREE, because it helps avoid data
> > corruption.
> > 
> > The new proposal tries to fix the TLB issue. We introduce two madvise verbs:
> > 
> > MARK_FREE. Userspace notifies kernel the memory range can be discarded. Kernel
> > just records the range in current stage. Should memory pressure happen, page
> > reclaim can free the memory directly regardless the pte state.
> > 
> > MARK_NOFREE. Userspace notifies kernel the memory range will be reused soon.
> > Kernel deletes the record and prevents page reclaim discards the memory. If the
> > memory isn't reclaimed, userspace will access the old memory, otherwise do
> > normal page fault handling.
> > 
> > The point is to let userspace notify kernel if memory can be discarded, instead
> > of depending on pte dirty bit used by MADV_FREE. With these, no TLB flush is
> > required till page reclaim actually frees the memory (page reclaim need do the
> > TLB flush for MADV_FREE too). It still preserves the lazy memory free merit of
> > MADV_FREE.
> > 
> > Compared to MADV_FREE, reusing memory with the new proposal isn't transparent,
> > eg must call MARK_NOFREE. But it's easy to utilize the new API in jemalloc.
> > 
> > We don't have code to backup this yet, sorry. We'd like to discuss it if it
> > makes sense.
> 
> It's really what volatile range did.
> John Stultz and me tried it for a *long* time but it had lots of troubles.
> It's really hard to write it down in my time due to really long history
> and even I forgot lots of detail(ie, dead brain).
> Please search volatile ranges in google.
> Finally, people in LSF/MM suggested MADV_FREE to help anonymous page side
> rather than stucking hich prevent useful feature. :(

I should have Cced John Stutlz.

He would have good memory than me so he would help but I'm not sure
he has a interest on volatile ranges, still.