From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60436) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1c7NT9-0006fY-Gu for qemu-devel@nongnu.org; Thu, 17 Nov 2016 09:15:16 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1c7NT8-0000Nq-F1 for qemu-devel@nongnu.org; Thu, 17 Nov 2016 09:15:15 -0500 Received: from mx1.redhat.com ([209.132.183.28]:47942) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1c7NT8-0000Mj-7A for qemu-devel@nongnu.org; Thu, 17 Nov 2016 09:15:14 -0500 Reply-To: krister@redhat.com References: <1479318422-10979-1-git-send-email-stefanha@redhat.com> From: Karl Rister Message-ID: <1206accf-f537-bc34-2e59-c13e220f870c@redhat.com> Date: Thu, 17 Nov 2016 08:15:11 -0600 MIME-Version: 1.0 In-Reply-To: <1479318422-10979-1-git-send-email-stefanha@redhat.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [PATCH v2 0/4] aio: experimental virtio-blk polling mode List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Stefan Hajnoczi , qemu-devel@nongnu.org Cc: borntraeger@de.ibm.com, Paolo Bonzini , Fam Zheng , Andrew Theurer I think these results look a bit more in line with expectations on the quick sniff test: QEMU_AIO_POLL_MAX_NS IOPs unset 26,299 1 25,929 2 25,753 4 27,214 8 27,053 16 26,861 32 24,752 64 25,058 128 24,732 256 25,560 512 24,614 1,024 25,186 2,048 25,829 4,096 25,671 8,192 27,896 16,384 38,086 32,768 35,493 65,536 38,496 131,072 38,296 I did a spot check of CPU utilization when the polling started having benefits. Without polling (QEMU_AIO_POLL_MAX_NS=unset) the iothread's CPU usage looked like this: user time: 25.94% system time: 22.11% With polling and QEMU_AIO_POLL_MAX_NS=16384 the iothread's CPU usage looked like this: user time: 78.92% system time: 20.80% Karl On 11/16/2016 11:46 AM, Stefan Hajnoczi wrote: > v2: > * Uninitialized node->deleted gone [Fam] > * Removed 1024 polling loop iteration qemu_clock_get_ns() optimization which > created a weird step pattern [Fam] > * Unified with AioHandler, dropped AioPollHandler struct [Paolo] > (actually I think Paolo had more in mind but this is the first step) > * Only poll when all event loop resources support it [Paolo] > * Added run_poll_handlers_begin/end trace events for perf analysis > * Sorry, Christian, no virtqueue kick suppression yet > > Recent performance investigation work done by Karl Rister shows that the > guest->host notification takes around 20 us. This is more than the "overhead" > of QEMU itself (e.g. block layer). > > One way to avoid the costly exit is to use polling instead of notification. > The main drawback of polling is that it consumes CPU resources. In order to > benefit performance the host must have extra CPU cycles available on physical > CPUs that aren't used by the guest. > > This is an experimental AioContext polling implementation. It adds a polling > callback into the event loop. Polling functions are implemented for virtio-blk > virtqueue guest->host kick and Linux AIO completion. > > The QEMU_AIO_POLL_MAX_NS environment variable sets the number of nanoseconds to > poll before entering the usual blocking poll(2) syscall. Try setting this > variable to the time from old request completion to new virtqueue kick. > > By default no polling is done. The QEMU_AIO_POLL_MAX_NS must be set to get any > polling! > > Stefan Hajnoczi (4): > aio: add AioPollFn and io_poll() interface > aio: add polling mode to AioContext > virtio: poll virtqueues for new buffers > linux-aio: poll ring for completions > > aio-posix.c | 115 ++++++++++++++++++++++++++++++++++++++++++-- > async.c | 14 +++++- > block/curl.c | 8 +-- > block/iscsi.c | 3 +- > block/linux-aio.c | 19 +++++++- > block/nbd-client.c | 8 +-- > block/nfs.c | 7 +-- > block/sheepdog.c | 26 +++++----- > block/ssh.c | 4 +- > block/win32-aio.c | 4 +- > hw/virtio/virtio.c | 18 ++++++- > include/block/aio.h | 8 ++- > iohandler.c | 2 +- > nbd/server.c | 9 ++-- > stubs/set-fd-handler.c | 1 + > tests/test-aio.c | 4 +- > trace-events | 4 ++ > util/event_notifier-posix.c | 2 +- > 18 files changed, 207 insertions(+), 49 deletions(-) > -- Karl Rister