Re: Ceph Hackathon: More Memory Allocator Testing

[Shinobu Kinjo <skinjo@redhat.com>]

Pre loading jemalloc after compiling with malloc

$ cat hoge.c 
#include <stdlib.h>

int main()
{
    int *ptr = malloc(sizeof(int) * 10);

    if (ptr == NULL)
        exit(EXIT_FAILURE);
    free(ptr);
}


$ gcc ./hoge.c


$ ldd ./a.out 
	linux-vdso.so.1 (0x00007fffe17e5000)
	libc.so.6 => /lib64/libc.so.6 (0x00007fc989c5f000)
	/lib64/ld-linux-x86-64.so.2 (0x000055a718762000)


$ nm ./a.out | grep malloc
                 U malloc@@GLIBC_2.2.5                       // malloc loaded


$ LD_PRELOAD=/usr/lib64/libjemalloc.so.1 \
> ldd a.out
	linux-vdso.so.1 (0x00007fff7fd36000)
	/usr/lib64/libjemalloc.so.1 (0x00007fe6ffe39000)    // jemallo loaded
	libc.so.6 => /lib64/libc.so.6 (0x00007fe6ffa61000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fe6ff844000)
	/lib64/ld-linux-x86-64.so.2 (0x0000560342ddf000)


Logically it could work, but in real world I'm not 100% sure if it works for large scale application.

Shinobu

----- Original Message -----
From: "Somnath Roy" <Somnath.Roy@sandisk.com>
To: "Alexandre DERUMIER" <aderumier@odiso.com>
Cc: "Sage Weil" <sage@newdream.net>, "Milosz Tanski" <milosz@adfin.com>, "Shishir Gowda" <Shishir.Gowda@sandisk.com>, "Stefan Priebe" <s.priebe@profihost.ag>, "Mark Nelson" <mnelson@redhat.com>, "ceph-devel" <ceph-devel@vger.kernel.org>
Sent: Sunday, August 23, 2015 2:03:41 AM
Subject: RE: Ceph Hackathon: More Memory Allocator Testing

Need to see if client is overriding the libraries built with different malloc libraries I guess..
I am not sure in your case the benefit you are seeing is because of qemu is more efficient with tcmalloc/jemalloc or the entire client stack ?

-----Original Message-----
From: Alexandre DERUMIER [mailto:aderumier@odiso.com] 
Sent: Saturday, August 22, 2015 9:57 AM
To: Somnath Roy
Cc: Sage Weil; Milosz Tanski; Shishir Gowda; Stefan Priebe; Mark Nelson; ceph-devel
Subject: Re: Ceph Hackathon: More Memory Allocator Testing

>>Wanted to know is there any reason we didn't link client libraries with tcmalloc at the first place (but did link only OSDs/mon/RGW) ?

Do we need to link client librairies ?

I'm building qemu with jemalloc , and it's seem to be enough.



----- Mail original -----
De: "Somnath Roy" <Somnath.Roy@sandisk.com>
À: "Sage Weil" <sage@newdream.net>, "Milosz Tanski" <milosz@adfin.com>
Cc: "Shishir Gowda" <Shishir.Gowda@sandisk.com>, "Stefan Priebe" <s.priebe@profihost.ag>, "aderumier" <aderumier@odiso.com>, "Mark Nelson" <mnelson@redhat.com>, "ceph-devel" <ceph-devel@vger.kernel.org>
Envoyé: Samedi 22 Août 2015 18:15:36
Objet: RE: Ceph Hackathon: More Memory Allocator Testing

Yes, even today rocksdb also linked with tcmalloc. It doesn't mean all the application using rocksdb needs to be built with tcmalloc. 
Sage,
Wanted to know is there any reason we didn't link client libraries with tcmalloc at the first place (but did link only OSDs/mon/RGW) ? 

Thanks & Regards
Somnath 

-----Original Message-----
From: Sage Weil [mailto:sage@newdream.net]
Sent: Saturday, August 22, 2015 6:56 AM
To: Milosz Tanski
Cc: Shishir Gowda; Somnath Roy; Stefan Priebe; Alexandre DERUMIER; Mark Nelson; ceph-devel
Subject: Re: Ceph Hackathon: More Memory Allocator Testing 

On Fri, 21 Aug 2015, Milosz Tanski wrote: 
> On Fri, Aug 21, 2015 at 12:22 AM, Shishir Gowda 
> <Shishir.Gowda@sandisk.com> wrote:
> > Hi All,
> > 
> > Have sent out a pull request which enables building librados/librbd with either tcmalloc(as default) or jemalloc. 
> > 
> > Please find the pull request @
> > https://github.com/ceph/ceph/pull/5628
> > 
> > With regards,
> > Shishir
> 
> Unless I'm missing something here, this seams like the wrong thing to. 
> Libraries that will be linked in by other external applications should 
> not have a 3rd party malloc linked in there. That seams like an 
> application choice. At the very least the default should not be to 
> link in a 3rd party malloc.

Yeah, I think you're right. 

Note that this isn't/wasn't always the case, though.. on precise, for instance, libleveldb links libtcmalloc. They stopped doing this sometime before trusty. 

sage 

________________________________ 

PLEASE NOTE: The information contained in this electronic mail message is intended only for the use of the designated recipient(s) named above. If the reader of this message is not the intended recipient, you are hereby notified that you have received this message in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify the sender by telephone or e-mail (as shown above) immediately and destroy any and all copies of this message in your possession (whether hard copies or electronically stored copies). 

N�����r��y���b�X��ǧv�^�)޺{.n�+���z�]z���{ay�ʇڙ�,j��f���h���z��w������j:+v���w�j�m��������zZ+��ݢj"��
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html