From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from [140.186.70.92] (port=47963 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1P1hES-0006YE-C7 for qemu-devel@nongnu.org; Fri, 01 Oct 2010 11:09:12 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1P1hAL-0006r8-5u for qemu-devel@nongnu.org; Fri, 01 Oct 2010 11:04:54 -0400 Received: from e33.co.us.ibm.com ([32.97.110.151]:58551) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1P1hAL-0006qx-0r for qemu-devel@nongnu.org; Fri, 01 Oct 2010 11:04:53 -0400 Received: from d03relay01.boulder.ibm.com (d03relay01.boulder.ibm.com [9.17.195.226]) by e33.co.us.ibm.com (8.14.4/8.13.1) with ESMTP id o91Exd2E022853 for ; Fri, 1 Oct 2010 08:59:39 -0600 Received: from d03av02.boulder.ibm.com (d03av02.boulder.ibm.com [9.17.195.168]) by d03relay01.boulder.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id o91F4kBg086946 for ; Fri, 1 Oct 2010 09:04:48 -0600 Received: from d03av02.boulder.ibm.com (loopback [127.0.0.1]) by d03av02.boulder.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id o91F4kZJ016853 for ; Fri, 1 Oct 2010 09:04:46 -0600 Message-ID: <4CA5F888.7010906@linux.vnet.ibm.com> Date: Fri, 01 Oct 2010 08:04:40 -0700 From: "Venkateswararao Jujjuri (JV)" MIME-Version: 1.0 Subject: Re: [Qemu-devel] QEMU throughput is down with SMP References: <4CA3DEC7.5070609@linux.vnet.ibm.com> <4CA4E2CA.5030606@linux.vnet.ibm.com> <20101001133855.GA30086@us.ibm.com> In-Reply-To: <20101001133855.GA30086@us.ibm.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Ryan Harper Cc: Stefan Hajnoczi , Qemu-development List On 10/1/2010 6:38 AM, Ryan Harper wrote: > * Stefan Hajnoczi [2010-10-01 03:48]: >> On Thu, Sep 30, 2010 at 8:19 PM, Venkateswararao Jujjuri (JV) >> wrote: >>> On 9/30/2010 2:13 AM, Stefan Hajnoczi wrote: >>>> >>>> On Thu, Sep 30, 2010 at 1:50 AM, Venkateswararao Jujjuri (JV) >>>> wrote: >>>>> >>>>> Code: Mainline QEMU (git://git.qemu.org/qemu.git) >>>>> Machine: LS21 blade. >>>>> Disk: Local disk through VirtIO. >>>>> Did not select any cache option. Defaulting to writethrough. >>>>> >>>>> Command tested: >>>>> 3 parallel instances of : dd if=/dev/zero of=/pmnt/my_pw bs=4k >>>>> count=100000 >>>>> >>>>> QEMU with smp=1 >>>>> 19.3 MB/s + 19.2 MB/s + 18.6 MB/s = 57.1 MB/s >>>>> >>>>> QEMU with smp=4 >>>>> 15.3 MB/s + 14.1 MB/s + 13.6 MB/s = 43.0 MB/s >>>>> >>>>> Is this expected? >>>> >>>> Did you configure with --enable-io-thread? >>> >>> Yes I did. >>>> >>>> Also, try using dd oflag=direct to eliminate effects introduced by the >>>> guest page cache and really hit the disk. >>> >>> With oflag=direct , I see no difference and the throughput is so slow and I >>> would not >>> expect to see any difference. >>> It is 225 kb/s for each thread either with smp=1 or with smp=4. >> >> If I understand correctly you are getting: >> >> QEMU oflag=direct with smp=1 >> 225 KB/s + 225 KB/s + 225 KB/s = 675 KB/s >> >> QEMU oflag=direct with smp=4 >> 225 KB/s + 225 KB/s + 225 KB/s = 675 KB/s >> >> This suggests the degradation for smp=4 is guest kernel page cache or >> buffered I/O related. Perhaps lockholder preemption? > > or just a single spindle maxed out because the blade hard drive doesn't > have writecache enabled (it's disabled by default). Yes, I am sure we are hitting the max limit on the blade local disk. Question is why the smp=4 degraded the performance in the cached mode. I am running latest kernel from upstream on the guest(2.6.36-rc5)..and using block IO. Do we have any know issues in there which could explain performance degradation? I am trying to get to a test which proves that the QEMU SMP improves/scales. I would like to use it in validating our new VirtFS threading code (yet to hit mailing list). Thanks, JV