Follow-Up on Alexandre's Transparent Huge Pages Testing

[Mark Nelson <mark.a.nelson@gmail.com>]

Hi Everyone,

A while back Alexandre Derumier posted some test results looking at how 
transparent huge pages can reduce memory usage with jemalloc.  I went 
back and ran a number of new tests on the community performance cluster 
to verify his findings and also look at how performance and cpu usage 
were affected, both during various fio benchmark tests and also during a 
4k random write recovery scenario.  I tested tcmalloc 2.4 with 32MB 
thread cache, 128MB thread cache, and jemalloc 4.0.

The gist of it is that I also see a reduction in memory usage, most 
pronounced with jemalloc.  Unfortuantely the best reduction in memory 
usage is when memory usage is already fairly low.  The most important 
case is the memory spike when OSDs are marked back up/in during a 
recovery test.  In this case there is still a benefit, though memory 
usage is still a little higher than TCMalloc with 128MB thread cache. 
There's a little bit of a concerning trend where memory usage appears to 
increase fairly quickly after the recovery test is complete and the 
post-recovery phase of the benchmark is running.  That will likely need 
to be investigate in more depth.

I have been doing some other tests with the async messenger and 
newstore, but those will have to wait for another paper.

Here's are the results:

https://drive.google.com/file/d/0B2gTBZrkrnpZY3U3TUU3RkJVeVk/view

Mark