From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:37207) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bNztU-0000Kn-8u for qemu-devel@nongnu.org; Fri, 15 Jul 2016 05:58:53 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bNztQ-0007Tg-4G for qemu-devel@nongnu.org; Fri, 15 Jul 2016 05:58:51 -0400 Received: from mail-lf0-x244.google.com ([2a00:1450:4010:c07::244]:36557) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bNztP-0007TZ-S2 for qemu-devel@nongnu.org; Fri, 15 Jul 2016 05:58:48 -0400 Received: by mail-lf0-x244.google.com with SMTP id 33so6809551lfw.3 for ; Fri, 15 Jul 2016 02:58:47 -0700 (PDT) Sender: Paolo Bonzini References: <20160713022334.GB16038@ad.usersys.redhat.com> <1468396629-26094-1-git-send-email-roman.penyaev@profitbricks.com> <20160713114520.GC5548@noname.redhat.com> From: Paolo Bonzini Message-ID: Date: Fri, 15 Jul 2016 11:58:43 +0200 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [PATCH V2 1/1] linux-aio: prevent submitting more than MAX_EVENTS List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Roman Penyaev , Kevin Wolf Cc: Fam Zheng , qemu-devel , Stefan Hajnoczi On 15/07/2016 11:18, Roman Penyaev wrote: > Those 3 red spikes and a blue hill is what we have to focus on. The > blue hill at the right corner of the chart means that almost always the > ring buffer was observed as full, i.e. qemu_laio_completion_bh() got > a chance to reap completions not very often, meanwhile completed > requests stand in the ring buffer for quite a long time which degrades > the overall performance. > > The results covered by the red line are much better and that can be > explained by those 3 red spikes, which are almost in the middle of the > whole distribution, i.e. qemu_laio_completion_bh() is called more often, > completed requests do not stall, giving fio a chance to submit new fresh > requests. > > The theoretical fix would be to schedule completion BH just after > successful io_submit, i.e.: What about removing the qemu_bh_cancel but keeping the rest of the patch? I'm also interested in a graph with this patch ("linux-aio: prevent submitting more than MAX_EVENTS") on top of origin/master. Thanks for the analysis. Sometimes a picture _is_ worth a thousand words, even if it's measuring "only" second-order effects (# of completions is not what causes the slowdown, but # of completions affects latency which causes the slowdown). Paolo