From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39385) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1X8AvX-000083-6v for qemu-devel@nongnu.org; Fri, 18 Jul 2014 12:22:40 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1X8AvO-0001jX-53 for qemu-devel@nongnu.org; Fri, 18 Jul 2014 12:22:31 -0400 Received: from mail1.windriver.com ([147.11.146.13]:49088) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1X8AvN-0001hk-SC for qemu-devel@nongnu.org; Fri, 18 Jul 2014 12:22:22 -0400 Message-ID: <53C949BA.9040204@windriver.com> Date: Fri, 18 Jul 2014 10:22:18 -0600 From: Chris Friesen MIME-Version: 1.0 References: <53C9362C.8040507@windriver.com> <53C93C34.7030403@redhat.com> In-Reply-To: <53C93C34.7030403@redhat.com> Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] is there a limit on the number of in-flight I/O operations? List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Paolo Bonzini , qemu-devel@nongnu.org On 07/18/2014 09:24 AM, Paolo Bonzini wrote: > Il 18/07/2014 16:58, Chris Friesen ha scritto: >> >> I've recently run up against an interesting issue where I had a number >> of guests running and when I started doing heavy disk I/O on a virtio >> disk (backed via ceph rbd) the memory consumption spiked and triggered >> the OOM-killer. >> >> I want to reserve some memory for I/O, but I don't know how much it can >> use in the worst-case. >> >> Is there a limit on the number of in-flight I/O operations? (Preferably >> as a configurable option, but even hard-coded would be good to know as >> well.) > > For rbd, there is no such limit in QEMU except the size of the virtio > ring buffer, but librbd may add limits of its own. > > For files, there's no limit if you use aio=threads, but the more I/O > operations you trigger the more threads QEMU will create. After 10 > seconds of idle the threads will be destroyed. > > Also for files, the limit is 128 per disk if you use aio=native. You > can only change it by recompilation only. Has anyone looked at enforcing some limits? I'm okay with throttling performance if necessary, but I really don't want to trigger the OOM-killer. I could do it locally, but it'd be nice to have some pointers as to where to start. Chris