From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:37151) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z6Bku-0000N2-Rf for qemu-devel@nongnu.org; Sat, 20 Jun 2015 01:55:55 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Z6Bkp-0002vz-5r for qemu-devel@nongnu.org; Sat, 20 Jun 2015 01:55:52 -0400 Received: from mailpro.odiso.net ([89.248.209.98]:33119) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z6Bko-0002v4-Tj for qemu-devel@nongnu.org; Sat, 20 Jun 2015 01:55:47 -0400 Date: Sat, 20 Jun 2015 07:55:44 +0200 (CEST) From: Alexandre DERUMIER Message-ID: <2143018002.1311772.1434779744377.JavaMail.zimbra@oxygem.tv> In-Reply-To: <1434711418-20429-1-git-send-email-aderumier@odiso.com> References: <1434711418-20429-1-git-send-email-aderumier@odiso.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [Qemu-devel] [PATCH] configure: Add support for jemalloc List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel Cc: Fam Zheng @cc Fam Zheng , as he's the author of tcmalloc support patch ----- Mail original ----- De: "aderumier" =C3=80: "qemu-devel" Cc: "aderumier" Envoy=C3=A9: Vendredi 19 Juin 2015 12:56:58 Objet: [PATCH] configure: Add support for jemalloc This adds "--enable-jemalloc" and "--disable-jemalloc" to allow linking=20 to jemalloc memory allocator.=20 We have already tcmalloc support,=20 but it seem to not working well with a lot of iothreads/disks.=20 The main problem is that tcmalloc use a shared thread cache of 16MB=20 by default.=20 With more threads, this cache is shared, and some bad garbage collections= =20 can occur if the cache is too low.=20 It's possible to tcmalloc cache increase it with a env var:=20 TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES=3D256MB=20 With default 16MB, performances are really bad with more than 2 disks.=20 Increasing to 256MB, it's helping but still have problem with 16 disks/ioth= reads.=20 Jemalloc don't have performance problem with default configuration.=20 Here the benchmark results in iops of 1 qemu vm randread 4K iodepth=3D32,= =20 with rbd block backend (librbd is doing a lot of memory allocation),=20 1 iothread by disk=20 glibc malloc=20 ------------=20 1 disk 29052=20 2 disks 55878=20 4 disks 127899=20 8 disks 240566=20 15 disks 269976=20 jemalloc=20 --------=20 1 disk 41278=20 2 disks 75781=20 4 disks 195351=20 8 disks 294241=20 15 disks 298199=20 tcmalloc 2.2.1 default 16M cache=20 --------------------------------=20 1 disk 37911=20 2 disks 67698=20 4 disks 41076=20 8 disks 43312=20 15 disks 37569=20 tcmalloc : 256M cache=20 ---------------------------=20 1 disk 33914=20 2 disks 58839=20 4 disks 148205=20 8 disks 213298=20 15 disks 218383=20 Signed-off-by: Alexandre Derumier =20 ---=20 configure | 29 +++++++++++++++++++++++++++++=20 1 file changed, 29 insertions(+)=20 diff --git a/configure b/configure=20 index 222694f..2fe1e05 100755=20 --- a/configure=20 +++ b/configure=20 @@ -336,6 +336,7 @@ vhdx=3D""=20 quorum=3D""=20 numa=3D""=20 tcmalloc=3D"no"=20 +jemalloc=3D"no"=20 # parse CC options first=20 for opt do=20 @@ -1147,6 +1148,10 @@ for opt do=20 ;;=20 --enable-tcmalloc) tcmalloc=3D"yes"=20 ;;=20 + --disable-jemalloc) jemalloc=3D"no"=20 + ;;=20 + --enable-jemalloc) jemalloc=3D"yes"=20 + ;;=20 *)=20 echo "ERROR: unknown option $opt"=20 echo "Try '$0 --help' for more information"=20 @@ -1420,6 +1425,8 @@ Advanced options (experts only):=20 --enable-numa enable libnuma support=20 --disable-tcmalloc disable tcmalloc support=20 --enable-tcmalloc enable tcmalloc support=20 + --disable-jemalloc disable jemalloc support=20 + --enable-jemalloc enable jemalloc support=20 NOTE: The object files are built at the place where configure is launched= =20 EOF=20 @@ -3344,6 +3351,11 @@ EOF=20 fi=20 fi=20 +if test "$tcmalloc" =3D "yes" && test "$jemalloc" =3D "yes" ; then=20 + echo "ERROR: tcmalloc && jemalloc can't be used at the same time"=20 + exit 1=20 +fi=20 +=20 ##########################################=20 # tcmalloc probe=20 @@ -3361,6 +3373,22 @@ EOF=20 fi=20 ##########################################=20 +# jemalloc probe=20 +=20 +if test "$jemalloc" =3D "yes" ; then=20 + cat > $TMPC << EOF=20 +#include =20 +int main(void) { malloc(1); return 0; }=20 +EOF=20 +=20 + if compile_prog "" "-ljemalloc" ; then=20 + LIBS=3D"-ljemalloc $LIBS"=20 + else=20 + feature_not_found "jemalloc" "install jemalloc devel"=20 + fi=20 +fi=20 +=20 +##########################################=20 # signalfd probe=20 signalfd=3D"no"=20 cat > $TMPC << EOF=20 @@ -4499,6 +4527,7 @@ echo "snappy support $snappy"=20 echo "bzip2 support $bzip2"=20 echo "NUMA host support $numa"=20 echo "tcmalloc support $tcmalloc"=20 +echo "jemalloc support $jemalloc"=20 if test "$sdl_too_old" =3D "yes"; then=20 echo "-> Your SDL version is too old - please upgrade to have SDL support"= =20 --=20 2.1.4=20