From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:51328) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1c6IcV-0008Lu-BY for qemu-devel@nongnu.org; Mon, 14 Nov 2016 09:52:28 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1c6IcU-0006Ny-Hh for qemu-devel@nongnu.org; Mon, 14 Nov 2016 09:52:27 -0500 Received: from mx1.redhat.com ([209.132.183.28]:57460) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1c6IcU-0006NW-CN for qemu-devel@nongnu.org; Mon, 14 Nov 2016 09:52:26 -0500 Received: from int-mx11.intmail.prod.int.phx2.redhat.com (int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 2C4E67AEA1 for ; Mon, 14 Nov 2016 14:52:24 +0000 (UTC) Reply-To: krister@redhat.com References: <1478711602-12620-1-git-send-email-stefanha@redhat.com> <5826231D.7070208@redhat.com> <20161114135317.GB2373@lemon> From: Karl Rister Message-ID: <5829CFA5.3060605@redhat.com> Date: Mon, 14 Nov 2016 08:52:21 -0600 MIME-Version: 1.0 In-Reply-To: <20161114135317.GB2373@lemon> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [RFC 0/3] aio: experimental virtio-blk polling mode List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Fam Zheng Cc: Stefan Hajnoczi , qemu-devel@nongnu.org, Paolo Bonzini , Andrew Theurer On 11/14/2016 07:53 AM, Fam Zheng wrote: > On Fri, 11/11 13:59, Karl Rister wrote: >> >> Stefan >> >> I ran some quick tests with your patches and got some pretty good gains, >> but also some seemingly odd behavior. >> >> These results are for a 5 minute test doing sequential 4KB requests from >> fio using O_DIRECT, libaio, and IO depth of 1. The requests are >> performed directly against the virtio-blk device (no filesystem) which >> is backed by a 400GB NVme card. >> >> QEMU_AIO_POLL_MAX_NS IOPs >> unset 31,383 >> 1 46,860 >> 2 46,440 >> 4 35,246 >> 8 34,973 >> 16 46,794 >> 32 46,729 >> 64 35,520 >> 128 45,902 > > For sequential read with ioq=1, each request takes >20000ns under 45,000 IOPs. > Isn't a poll time of 128ns a mismatching order of magnitude? Have you tried > larger values? Not criticizing, just trying to understand how it workd. Not yet, I was just trying to get something out as quick as I could (while juggling this with some other stuff...). Frankly I was a bit surprised that the low values made such an impact and then got distracted by the behaviors of 4, 8, and 64. > > Also, do you happen to have numbers for unpatched QEMU (just to confirm that > "unset" case doesn't cause regression) and baremetal for comparison? I didn't run this exact test on the same qemu.git master changeset unpatched. I did however previously try it against the v2.7.0 tag and got somewhere around 27.5K IOPs. My original intention was to apply the patches to v2.7.0 but it wouldn't build. We have done a lot of testing and tracing on the qemu-rhev package and 27K IOPs is about what we see there (with tracing disabled). Given the patch discussions I saw I was mainly trying to get a sniff test out and then do a more complete workup with whatever updates are made. I should probably note that there are a lot of pinning optimizations made here to assist in our tracing efforts which also result in improved performance. Ultimately, in a proper evaluation of these patches most of that will be removed so the behavior may change somewhat. > > Fam > -- Karl Rister