Re: Ceph Hackathon: More Memory Allocator Testing

[Mark Nelson <mnelson@redhat.com>]

On 08/19/2015 07:36 AM, Dałek, Piotr wrote:
>> -----Original Message-----
>> From: ceph-devel-owner@vger.kernel.org [mailto:ceph-devel-
>> owner@vger.kernel.org] On Behalf Of Mark Nelson
>> Sent: Wednesday, August 19, 2015 2:17 PM
>>
>>> The RSS memory usage in the report is per OSD I guess(really?). It
>>> can't be ignored since it's really a great improvement memory usage.
>>
>> Do you mean with tcmalloc?  I think it's a tough decision.  For jemalloc, 300MB
>> more of RSS per OSD does add up (about 18GB for 60 OSDs).  On the other
>> hand, the cost of memory is such a small fraction of the overall cost of
>> systems like this that it might be worth it to switch over anyway.  In the 4K
>> write tests it's pretty clear that even with 128MB TC, TCMalloc is suffering
>> and jemalloc appears to still have headroom left.  It's possible that bumping
>> the thread cache even higher might help TCMalloc close the gap though.  It's
>> also possible that jemalloc might have worse memory behavior under
>> recovery scenarios as we discussed at the hackathon (And Somnath
>> mentioned above), so I think we probably need to run the tests.
>
> Have you tried running these tests again with TCMalloc after applying patches from https://github.com/ceph/ceph/pull/5534? Because switching to jemalloc alone won't fix the root issues, only make it less painful.

Not yet, my first goal was to simply replicate the tests that have 
already been done using hammer to get a baseline on the new cluster.  It 
definitely looks like that's (yet another!) good next step.

Mark

>
> With best regards / Pozdrawiam
> Piotr Dałek
>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html