From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38308) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Zzqk1-0006u7-0H for qemu-devel@nongnu.org; Fri, 20 Nov 2015 13:49:01 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Zzqjx-0004d7-PR for qemu-devel@nongnu.org; Fri, 20 Nov 2015 13:49:00 -0500 Received: from mx1.redhat.com ([209.132.183.28]:42900) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Zzqjx-0004cG-HX for qemu-devel@nongnu.org; Fri, 20 Nov 2015 13:48:57 -0500 Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) by mx1.redhat.com (Postfix) with ESMTPS id 131B95BA02 for ; Fri, 20 Nov 2015 18:48:57 +0000 (UTC) References: <1448010269-21694-1-git-send-email-pbonzini@redhat.com> <1448010269-21694-5-git-send-email-pbonzini@redhat.com> From: Eric Blake Message-ID: <564F6B17.5040201@redhat.com> Date: Fri, 20 Nov 2015 11:48:55 -0700 MIME-Version: 1.0 In-Reply-To: <1448010269-21694-5-git-send-email-pbonzini@redhat.com> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="7DaJbJeAe3fHQN5Mgf6g69RDBURvBS7US" Subject: Re: [Qemu-devel] [PATCH 4/4] qjson: surprise, allocating 6 QObjects per token is expensive List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Paolo Bonzini , qemu-devel@nongnu.org Cc: armbru@redhat.com This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --7DaJbJeAe3fHQN5Mgf6g69RDBURvBS7US Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 11/20/2015 02:04 AM, Paolo Bonzini wrote: > Replace the contents of the tokens GQueue with a simple struct. This c= uts > the amount of memory allocated by tests/check-qjson from ~500MB to ~20M= B, > and the execution time from 600ms to 80ms on my laptop. Still a lot (s= ome > could be saved by using an intrusive list, such as QSIMPLEQ, instead of= > the GQueue), but the savings are already massive and the right thing to= > do would probably be to get rid of json-streamer completely. >=20 > Signed-off-by: Paolo Bonzini > --- > include/qapi/qmp/json-streamer.h | 7 ++++ > qobject/json-parser.c | 81 +++++++++++++++++++-------------= -------- > qobject/json-streamer.c | 19 ++++------ > 3 files changed, 53 insertions(+), 54 deletions(-) >=20 > @@ -54,15 +50,16 @@ static void json_message_process_token(JSONLexer *l= exer, GString *input, JSONTok > } > } > =20 > - dict =3D qdict_new(); > - qdict_put(dict, "type", qint_from_int(type)); > - qdict_put(dict, "token", qstring_from_str(input->str)); > - qdict_put(dict, "x", qint_from_int(x)); > - qdict_put(dict, "y", qint_from_int(y)); > + token =3D g_malloc(sizeof(JSONToken) + input->len + 1); > + token->type =3D type; > + memcpy(token->str, input->str, input->len + 1); > + token->str[input->len] =3D 0; Looks like you are writing the last byte twice. Either the +1 in the memcpy() always copies a NUL byte and we don't need the second assignment, or you should drop the +1. Otherwise, looks like a sane replacement that saves a lot of memory. Reviewed-by: Eric Blake Are we hoping to get this in 2.5 because it fixes the memory hog bug, or are we considering that it is not a regression from 2.4 and therefore something that should wait for 2.6? --=20 Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org --7DaJbJeAe3fHQN5Mgf6g69RDBURvBS7US Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 Comment: Public key at http://people.redhat.com/eblake/eblake.gpg Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBCAAGBQJWT2sXAAoJEKeha0olJ0Nq/IoH/2GnCaErwX1X+6WWxvXPbKcI eJJlCo6u/Cut15mQEClGhhH6mShwCQ97IZy89gp3mHsnNDCC5BJTScWSgosIi5qS Sd/hrzJii7jepN99kRy26RZvXbakal4zlRMkmgSk4faLFaaTQ6oGGidOmNCYwkCM kdDlzq7HTqrsxUzlKg5Od9U1WPzDseP6Vy2Z3rN7ffsBUoNZkSjsraKdWgxah7fW C6YgwCxO50YBU/Jj280zX2LNCjFlPSQUtAvTxsQPSz75jl2XEgY6/XcD4vuoRcxB EvjFBVRrncIHeNiGFm5CqYah6nF9xZDuQYj9fgE7qt+2Rbi3YO4105FoSfSfAzY= =UoTG -----END PGP SIGNATURE----- --7DaJbJeAe3fHQN5Mgf6g69RDBURvBS7US--