From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from [140.186.70.92] (port=45918 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1P1fpR-000483-3k for qemu-devel@nongnu.org; Fri, 01 Oct 2010 09:39:19 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1P1fpG-0007w6-48 for qemu-devel@nongnu.org; Fri, 01 Oct 2010 09:39:12 -0400 Received: from e6.ny.us.ibm.com ([32.97.182.146]:47354) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1P1fpG-0007u9-1p for qemu-devel@nongnu.org; Fri, 01 Oct 2010 09:39:02 -0400 Received: from d01relay03.pok.ibm.com (d01relay03.pok.ibm.com [9.56.227.235]) by e6.ny.us.ibm.com (8.14.4/8.13.1) with ESMTP id o91DdDa3008538 for ; Fri, 1 Oct 2010 09:39:13 -0400 Received: from d01av01.pok.ibm.com (d01av01.pok.ibm.com [9.56.224.215]) by d01relay03.pok.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id o91DcwO4348532 for ; Fri, 1 Oct 2010 09:38:58 -0400 Received: from d01av01.pok.ibm.com (loopback [127.0.0.1]) by d01av01.pok.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id o91DcwZa006546 for ; Fri, 1 Oct 2010 09:38:58 -0400 Date: Fri, 1 Oct 2010 08:38:55 -0500 From: Ryan Harper Subject: Re: [Qemu-devel] QEMU throughput is down with SMP Message-ID: <20101001133855.GA30086@us.ibm.com> References: <4CA3DEC7.5070609@linux.vnet.ibm.com> <4CA4E2CA.5030606@linux.vnet.ibm.com> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline In-Reply-To: Content-Transfer-Encoding: quoted-printable List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Stefan Hajnoczi Cc: "Venkateswararao Jujjuri (JV)" , Qemu-development List * Stefan Hajnoczi [2010-10-01 03:48]: > On Thu, Sep 30, 2010 at 8:19 PM, Venkateswararao Jujjuri (JV) > wrote: > > On 9/30/2010 2:13 AM, Stefan Hajnoczi wrote: > >> > >> On Thu, Sep 30, 2010 at 1:50 AM, Venkateswararao Jujjuri (JV) > >> =A0wrote: > >>> > >>> Code: Mainline QEMU (git://git.qemu.org/qemu.git) > >>> Machine: LS21 blade. > >>> Disk: Local disk through VirtIO. > >>> Did not select any cache option. Defaulting to writethrough. > >>> > >>> Command tested: > >>> 3 parallel instances of : dd if=3D/dev/zero of=3D/pmnt/my_pw bs=3D4= k > >>> count=3D100000 > >>> > >>> QEMU with =A0smp=3D1 > >>> 19.3 MB/s + 19.2 MB/s + 18.6 MB/s =3D 57.1 MB/s > >>> > >>> QEMU with smp=3D4 > >>> 15.3 MB/s + 14.1 MB/s + 13.6 MB/s =3D 43.0 MB/s > >>> > >>> Is this expected? > >> > >> Did you configure with --enable-io-thread? > > > > Yes I did. > >> > >> Also, try using dd oflag=3Ddirect to eliminate effects introduced by= the > >> guest page cache and really hit the disk. > > > > With oflag=3Ddirect , I see no difference and the throughput is so sl= ow and I > > would not > > expect to see any difference. > > It is 225 kb/s =A0for each thread either with smp=3D1 or with smp=3D4. >=20 > If I understand correctly you are getting: >=20 > QEMU oflag=3Ddirect with=A0smp=3D1 > 225 KB/s + 225 KB/s + 225 KB/s =3D 675 KB/s >=20 > QEMU oflag=3Ddirect with smp=3D4 > 225 KB/s + 225 KB/s + 225 KB/s =3D 675 KB/s >=20 > This suggests the degradation for smp=3D4 is guest kernel page cache or > buffered I/O related. Perhaps lockholder preemption? or just a single spindle maxed out because the blade hard drive doesn't have writecache enabled (it's disabled by default). =20 --=20 Ryan Harper Software Engineer; Linux Technology Center IBM Corp., Austin, Tx ryanh@us.ibm.com