From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:51216) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1X8GxJ-0004s5-MS for qemu-devel@nongnu.org; Fri, 18 Jul 2014 18:48:54 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1X8GxB-0003ON-F7 for qemu-devel@nongnu.org; Fri, 18 Jul 2014 18:48:45 -0400 Received: from mail.windriver.com ([147.11.1.11]:47635) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1X8GxB-0003OF-7L for qemu-devel@nongnu.org; Fri, 18 Jul 2014 18:48:37 -0400 Message-ID: <53C9A440.7020306@windriver.com> Date: Fri, 18 Jul 2014 16:48:32 -0600 From: Chris Friesen MIME-Version: 1.0 References: <53C9362C.8040507@windriver.com> <53C93C34.7030403@redhat.com> <53C949BA.9040204@windriver.com> <53C97FEB.9060208@redhat.com> In-Reply-To: <53C97FEB.9060208@redhat.com> Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] is there a limit on the number of in-flight I/O operations? List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Paolo Bonzini , qemu-devel@nongnu.org On 07/18/2014 02:13 PM, Paolo Bonzini wrote: > Il 18/07/2014 18:22, Chris Friesen ha scritto: >> On 07/18/2014 09:24 AM, Paolo Bonzini wrote: >>> Il 18/07/2014 16:58, Chris Friesen ha scritto: >>>> >>>> I've recently run up against an interesting issue where I had a number >>>> of guests running and when I started doing heavy disk I/O on a virtio >>>> disk (backed via ceph rbd) the memory consumption spiked and triggered >>>> the OOM-killer. >>>> >>>> I want to reserve some memory for I/O, but I don't know how much it can >>>> use in the worst-case. >>>> >>>> Is there a limit on the number of in-flight I/O operations? (Preferably >>>> as a configurable option, but even hard-coded would be good to know as >>>> well.) >>> >>> For rbd, there is no such limit in QEMU except the size of the virtio >>> ring buffer, but librbd may add limits of its own. >>> >>> For files, there's no limit if you use aio=threads, but the more I/O >>> operations you trigger the more threads QEMU will create. After 10 >>> seconds of idle the threads will be destroyed. >>> >>> Also for files, the limit is 128 per disk if you use aio=native. You >>> can only change it by recompilation only. >> >> Has anyone looked at enforcing some limits? I'm okay with throttling >> performance if necessary, but I really don't want to trigger the >> OOM-killer. > > I forgot about "-drive ...,iops_max=NNN". :) I'm not sure it's actually useful though, since it specifies the max IO operations per second, not the maximum number of in-flight operations. If the far end can't keep up with the requested rate of operations, then the number of in-flight operations is going to grow even if you throttle the number of operations per second. Chris