From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60870) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XKzgS-0007W6-BA for qemu-devel@nongnu.org; Fri, 22 Aug 2014 21:00:04 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1XKzgK-0007mV-3L for qemu-devel@nongnu.org; Fri, 22 Aug 2014 20:59:56 -0400 Received: from mail.windriver.com ([147.11.1.11]:36287) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XKzgJ-0007mI-Qu for qemu-devel@nongnu.org; Fri, 22 Aug 2014 20:59:48 -0400 Message-ID: <53F7E77A.9050509@windriver.com> Date: Fri, 22 Aug 2014 18:59:38 -0600 From: Chris Friesen MIME-Version: 1.0 References: <53C949BA.9040204@windriver.com> <53C97FEB.9060208@redhat.com> <53C9A440.7020306@windriver.com> <53CA06ED.1090102@redhat.com> <53CA0FC4.8080802@windriver.com> <53CA1D06.9090601@redhat.com> <20140719084537.GA3058@irqsave.net> <53CD2AE1.6080803@windriver.com> <20140721151540.GA22161@irqsave.net> <53CD3341.60705@windriver.com> <20140721161034.GC22161@irqsave.net> In-Reply-To: <20140721161034.GC22161@irqsave.net> Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: quoted-printable Subject: Re: [Qemu-devel] is there a limit on the number of in-flight I/O operations? List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: =?ISO-8859-1?Q?Beno=EEt_Canet?= Cc: Paolo Bonzini , qemu-devel@nongnu.org On 07/21/2014 10:10 AM, Beno=EEt Canet wrote: > The Monday 21 Jul 2014 =E0 09:35:29 (-0600), Chris Friesen wrote : >> On 07/21/2014 09:15 AM, Beno=EEt Canet wrote: >>> The Monday 21 Jul 2014 =E0 08:59:45 (-0600), Chris Friesen wrote : >>>> On 07/19/2014 02:45 AM, Beno=EEt Canet wrote: >>>> >>>>> I think in the throttling case the number of in flight operation is= limited by >>>>> the emulated hardware queue. Else request would pile up and throttl= ing would be >>>>> inefective. >>>>> >>>>> So this number should be around: #define VIRTIO_PCI_QUEUE_MAX 64 or= something like than that. >>>> >>>> Okay, that makes sense. Do you know how much data can be written as= part of >>>> a single operation? We're using 2MB hugepages for the guest memory,= and we >>>> saw the qemu RSS numbers jump from 25-30MB during normal operation u= p to >>>> 120-180MB when running dbench. I'd like to know what the worst-case= would > > Sorry I didn't understood this part at first read. > > In the linux guest can you monitor: > benoit@Laure:~$ cat /sys/class/block/xyz/inflight ? > > This would give us a faily precise number of the requests actually in f= light between the guest and qemu. After a bit of a break I'm looking at this again. While doing "dd if=3D/dev/zero of=3Dtestfile bs=3D1M count=3D700" in the = guest,=20 I got a max "inflight" value of 181. This seems quite a bit higher than=20 VIRTIO_PCI_QUEUE_MAX. I've seen throughput as high as ~210 MB/sec, which also kicked the RSS=20 numbers up above 200MB. I tried dropping VIRTIO_PCI_QUEUE_MAX down to 32 (it didn't seem to work=20 at all for values much less than that, though I didn't bother getting an=20 exact value) and it didn't really make any difference, I saw inflight=20 values as high as 177. Chris